mirror_php/parallel
1
0
Fork 0
mirror of https://github.com/amphp/parallel.git synced 2025-01-29 10:17:56 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity

Move documentation to README

Browse Source
This commit is contained in:
Niklas Keller 2022-10-03 15:34:35 +02:00
parent 2f9cb616ee
commit 54bfb340ce
No known key found for this signature in database
GPG Key ID: AFA536ABA90C76A6
11 changed files with 86 additions and 354 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
.gitmodules vendored
View File

@ -1,3 +0,0 @@
[submodule "docs/.shared"]
path = docs/.shared
url = https://github.com/amphp/amphp.github.io

87
README.md
View File

@ -48,7 +48,92 @@ foreach ($responses as $url => $response) {
`FetchTask` is just used as an example for a blocking function here.
If you just want to fetch multiple HTTP resources concurrently, it's better to use [`amphp/http-client`](https://amphp.org/http-client/), our non-blocking HTTP client.
The functions you call must be predefined or autoloadable by Composer, so they also exist in the worker processes.
> **Note**
> The functions you call must be predefined or autoloadable by Composer, so they also exist in the worker processes.
## Workers
`Worker` provides a simple interface for executing PHP code in parallel in a separate PHP process or thread.
Classes implementing [`Task`](#tasks) are used to define the code to be run in parallel.
## Tasks
The `Task` interface has a single `run()` method that gets invoked in the worker to dispatch the work that needs to be done.
The `run()` method can be written using blocking code since the code is executed in a separate process or thread.
Task instances are `serialize`'d in the main process and `unserialize`'d in the worker.
That means that all data that is passed between the main process and a worker needs to be serializable.
## Worker Pools
The easiest way to use workers is through a worker pool.
Worker pools can be used to submit tasks in the same way as a worker, but rather than using a single worker process or thread, the pool uses multiple workers to execute tasks.
This allows multiple tasks to be executed simultaneously.
The `WorkerPool` interface extends [`Worker`](#workers), adding methods to get information about the pool or pull a single `Worker` instance out of the pool.
A pool uses multiple `Worker` instances to execute [`Task`](#tasks) instances.
If a set of tasks should be run within a single worker, use the `WorkerPool::getWorker()` method to pull a single worker from the pool.
The worker is automatically returned to the pool when the instance returned is destroyed.
### Global Worker Pool
A global worker pool is available and can be set using the function `Amp\Parallel\Worker\workerPool(?WorkerPool $pool = null)`.
Passing an instance of `WorkerPool` will set the global pool to the given instance.
Invoking the function without an instance will return the current global instance.
## Processes and Threads
The `Process` and `Parallel` classes simplify writing and running PHP in parallel.
A script written to be run in parallel must return a callable that will be run in a child process (or a thread if [`ext-parallel`](https://github.com/krakjoe/parallel) is installed).
The callable receives a single argument – an instance of `Channel` that can be used to send data between the parent and child processes. Any serializable data can be sent across this channel.
The `Context` object, which extends the `Channel` interface, is the other end of the communication channel.
In the example below, a child process or thread is used to call a blocking function (`file_get_contents()` is only an example of a blocking function, use [`http-client`](https://amphp.org/http-client) for non-blocking HTTP requests).
The result of that function is then sent back to the parent using the `Channel` object.
The return value of the child process callable is available using the `Context::join()` method.
### Child Process or Thread
```php
# child.php
use Amp\Parallel\Sync\Channel;
return function (Channel $channel): mixed {
$url = $channel->receive();
$data = file_get_contents($url); // Example blocking function
$channel->send($data);
return 'Any serializable data';
};
```
### Parent Process
```php
# parent.php
use Amp\Parallel\Context;
// Creates a context using Process, or if ext-parallel is installed, Parallel.
$context = Context\createContext(__DIR__ . '/child.php');
$pid = $context->start();
$url = 'https://google.com';
$context->send($url);
$requestData = $context->receive();
printf("Received %d bytes from %s\n", \strlen($requestData), $url);
$returnValue = $context->join();
printf("Child processes exited with '%s'\n", $returnValue);
```
Child processes are also great for CPU-intensive operations such as image manipulation or for running daemons that perform periodic tasks based on input from the parent.
## Versioning

2
docs/.gitignore vendored
View File

@ -1,2 +0,0 @@
.bundle
_site

1
docs/.shared

@ -1 +0,0 @@
Subproject commit a965e8592848f75cb6f10b0acd2c73276bd1c04e

5
docs/Gemfile
View File

@ -1,5 +0,0 @@
source "https://rubygems.org"
gem "github-pages"
gem "kramdown"
gem "jekyll-github-metadata"
gem "jekyll-relative-links"

29
docs/_config.yml
View File

@ -1,29 +0,0 @@
kramdown:
input: GFM
toc_levels: 2..3
baseurl: "/parallel"
layouts_dir: ".shared/layout"
includes_dir: ".shared/includes"
exclude: ["Gemfile", "Gemfile.lock", "README.md", "vendor"]
safe: true
repository: amphp/parallel
gems:
- "jekyll-github-metadata"
- "jekyll-relative-links"
defaults:
- scope:
path: ""
type: "pages"
values:
layout: "docs"
shared_asset_path: "/parallel/asset"
navigation:
- processes
- workers
- worker-pool

1
docs/asset
View File

@ -1 +0,0 @@
.shared/asset

49
docs/index.md
View File

@ -1,49 +0,0 @@
---
title: Parallel processing for PHP
permalink: /
---
This package provides *true parallel processing* for PHP using multiple processes or native threads, *without blocking and no extensions required*.
## Installation
This package can be installed as a [Composer](https://getcomposer.org/) dependency.
```bash
composer require amphp/parallel
```
## Usage
The basic usage of this library is to submit blocking tasks to be executed by a worker pool in order to avoid blocking the main event loop.
```php
<?php
require __DIR__ . '/../vendor/autoload.php';
use Amp\Parallel\Worker;
use Amp\Promise;
$urls = [
'https://secure.php.net',
'https://amphp.org',
'https://github.com',
];
$promises = [];
foreach ($urls as $url) {
$promises[$url] = Worker\enqueueCallable('file_get_contents', $url);
}
$responses = Promise\wait(Promise\all($promises));
foreach ($responses as $url => $response) {
\printf("Read %d bytes from %s\n", \strlen($response), $url);
}
```
[`file_get_contents`](https://secure.php.net/file_get_contents) is just used as an example for a blocking function here.
If you just want to fetch multiple HTTP resources concurrently, it's better to use [`amphp/http-client`](https://amphp.org/http-client/), our non-blocking HTTP client.
The functions you call must be predefined or autoloadable by Composer, so they also exist in the worker processes.
Instead of simple callables, you can also [submit `Task` instances](./workers#task) with `Amp\Parallel\Worker\submit()`.

55
docs/processes.md
View File

@ -1,55 +0,0 @@
---
title: Processes and Threads
permalink: /processes
---
The `Process` and `Parallel` classes simplify writing and running PHP in parallel. A script written to be run in parallel must return a callable that will be run in a child process (or a thread if [`ext-parallel`](https://github.com/krakjoe/parallel) is installed). The callable receives a single argument – an instance of `Channel` that can be used to send data between the parent and child processes. Any serializable data can be sent across this channel. The `Context` object, which extends the `Channel` interface, is the other end of the communication channel.
In the example below, a child process or thread is used to call a blocking function (`file_get_contents()` is only an example of a blocking function, use [`http-client`](https://amphp.org/http-client) for non-blocking HTTP requests). The result of that function is then sent back to the parent using the `Channel` object. The return value of the child process callable is available using the `Context::join()` method.
## Child process or thread
```php
# child.php
use Amp\Parallel\Sync\Channel;
return function (Channel $channel): \Generator {
$url = yield $channel->receive();
$data = file_get_contents($url); // Example blocking function
yield $channel->send($data);
return 'Any serializable data';
};
```
## Parent Process
```php
# parent.php
use Amp\Loop;
use Amp\Parallel\Context;
Loop::run(function () {
// Creates a context using Process, or if ext-parallel is installed, Parallel.
$context = Context\createContext(__DIR__ . '/child.php');
$pid = yield $context->start();
$url = 'https://google.com';
yield $context->send($url);
$requestData = yield $context->receive();
printf("Received %d bytes from %s\n", \strlen($requestData), $url);
$returnValue = yield $context->join();
printf("Child processes exited with '%s'\n", $returnValue);
});
```
Child processes are also great for CPU-intensive operations such as image manipulation or for running daemons that perform periodic tasks based on input from the parent.

67
docs/worker-pool.md
View File

@ -1,67 +0,0 @@
---
title: Worker Pool
permalink: /worker-pool
---
The easiest way to use workers is through a worker pool. `Pool` implements `Worker`, so worker pools can be used to submit
tasks in the same way as a worker, but rather than using a single worker process or thread, the pool uses multiple workers
to execute tasks. This allows multiple tasks to be executed simultaneously.
## `WorkerPool`
The `WorkerPool` interface extends [`Worker`](./workers#worker), adding methods to get information about the pool or pull a single `Worker` instance
out of the pool. A pool uses multiple `Worker` instances to execute [tasks](./workers#task).
```php
<?php
namespace Amp\Parallel\Worker;
/**
* An interface for worker pools.
*/
interface Pool extends Worker
{
/** @var int The default maximum pool size. */
const DEFAULT_MAX_SIZE = 32;
/**
* Gets a worker from the pool. The worker is marked as busy and will only be reused if the pool runs out of
* idle workers. The worker will be automatically marked as idle once no references to the returned worker remain.
*
* @return Worker
*
* @throws \Amp\Parallel\Context\StatusError If the queue is not running.
*/
public function getWorker(): Worker;
/**
* Gets the number of workers currently running in the pool.
*
* @return int The number of workers.
*/
public function getWorkerCount(): int;
/**
* Gets the number of workers that are currently idle.
*
* @return int The number of idle workers.
*/
public function getIdleWorkerCount(): int;
/**
* Gets the maximum number of workers the pool may spawn to handle concurrent tasks.
*
* @return int The maximum number of workers.
*/
public function getMaxSize(): int;
}
```
If a set of tasks should be run within a single worker, use the `Pool::getWorker()` method to pull a single worker from the pool.
The worker is automatically returned to the pool when the instance returned is destroyed.
### Global worker pool
A global worker pool is available and can be set using the function `Amp\Parallel\Worker\pool(?Pool $pool = null)`.
Passing an instance of `Pool` will set the global pool to the given instance. Invoking the function without an instance will return
the current global instance.

141
docs/workers.md
View File

@ -1,141 +0,0 @@
---
title: Workers
permalink: /workers
---
## `Worker`
`Worker` provides a simple interface for executing PHP code in parallel in a separate PHP process or thread.
Classes implementing [`Task`](#task) are used to define the code to be run in parallel.
```php
<?php
namespace Amp\Parallel\Worker;
use Amp\Cancellation;
/**
* An interface for a parallel worker thread that runs a queue of tasks.
*/
interface Worker
{
/**
* Checks if the worker is running.
*
* @return bool True if the worker is running, otherwise false.
*/
public function isRunning(): bool;
/**
* Checks if the worker is currently idle.
*
* @return bool
*/
public function isIdle(): bool;
/**
* @template TResult
* @template TReceive
* @template TSend
* @template TCache
*
* Executes a {@see Task} on the worker.
*
* @param Task<TResult, TReceive, TSend, TCache> $task The task to execute.
* @param Cancellation|null $cancellation Token to request cancellation. The task must support cancellation for
* this to have any effect.
*
* @return Execution<TResult, TReceive, TSend, TCache>
*/
public function submit(Task $task, ?Cancellation $cancellation = null): Execution;
/**
* Gracefully shutdown the worker once all outstanding tasks have completed executing. Returns once the
* worker has been shutdown.
*/
public function shutdown(): void;
/**
* Immediately kills the context.
*/
public function kill(): void;
}
```
## `Task`
The `Task` interface has a single `run()` method that gets invoked in the worker to dispatch the work that needs to be done.
The `run()` method can be written using blocking code since the code is executed in a separate process or thread. The method
may also be asynchronous, returning a `Promise` or `Generator` that is run as a coroutine.
```php
<?php
namespace Amp\Parallel\Worker;
/**
* A runnable unit of execution.
*/
interface Task
{
/**
* Runs the task inside the caller's context.
*
* Does not have to be a coroutine, can also be a regular function returning a value.
*
* @param Environment
*
* @return mixed|\Amp\Promise|\Generator
*/
public function run(Environment $environment);
}
```
Task instances are `serialize`'d in the main process and `unserialize`'d in the worker.
That means that all data that is passed between the main process and a worker needs to be serializable.
## `Environment`
The passed `Environment` allows to persist data between multiple tasks executed by the same worker, e.g. database connections or file handles, without resorting to globals for that.
Additionally `Environment` allows setting a TTL for entries, so can be used as a cache.
```php
<?php
namespace Amp\Parallel\Worker;
interface Environment extends \ArrayAccess
{
/**
* @param string $key
*
* @return bool
*/
public function exists(string $key): bool;
/**
* @param string $key
*
* @return mixed|null Returns null if the key does not exist.
*/
public function get(string $key);
/**
* @param string $key
* @param mixed $value Using null for the value deletes the key.
* @param int $ttl Number of seconds until data is automatically deleted. Use null for unlimited TTL.
*/
public function set(string $key, $value, int $ttl = null);
/**
* @param string $key
*/
public function delete(string $key);
/**
* Removes all values.
*/
public function clear();
}
```