mirror of
https://github.com/themsaid/ibis.git
synced 2025-01-17 21:49:17 +01:00
95 lines
5.0 KiB
Markdown
95 lines
5.0 KiB
Markdown
## Handling Queues on Deployments
|
|
|
|
>{notice} This is a sample content from [Laravel Queues in Action](https://learn-laravel-queues.com/). A book by [Mohamed Said](https://twitter.com/themsaid) the creator of Ibis.
|
|
|
|
When you deploy your application with new code or different configurations, workers running on your servers need to be informed about the changes. Since workers are long-living processes, they must be shut down and restarted in order for the changes to be reflected.
|
|
|
|
### Restarting Workers Through The CLI
|
|
|
|
When writing your deployment script, you need to run the following command after pulling the new changes:
|
|
|
|
```php
|
|
php artisan queue:restart
|
|
```
|
|
|
|
This command will send a signal to all running workers instructing them to exit after finishing any job in hand. This is called "graceful termination".
|
|
|
|
If you're using Laravel Forge, here's a typical deployment script that you may use:
|
|
|
|
```bash
|
|
cd /home/forge/mysite.com
|
|
git pull origin master
|
|
$FORGE_COMPOSER install --no-interaction --prefer-dist --optimize-autoloader
|
|
|
|
( flock -w 10 9 || exit 1
|
|
echo 'Restarting FPM...'; sudo -S service $FORGE_PHP_FPM reload ) 9>/tmp/fpmlock
|
|
|
|
$FORGE_PHP artisan migrate --force
|
|
$FORGE_PHP artisan queue:restart
|
|
```
|
|
|
|
Here the new code will be pulled from git, dependencies will be installed by composer, php-fpm will be restarted, migrations will run, and finally, the queue restart signal will be sent.
|
|
|
|
After `php-fpm` is restarted, your application visitors will start using the new code while the workers are still running on older code. Eventually, those workers will exit and be started again by Supervisor. The new worker processes will be running the new code.
|
|
|
|
If you're using Envoyer, then you need to add a deployment hook after the “Activate New Release” action and run the `queue:restart` command.
|
|
|
|
### Restarting Workers Through Supervisor
|
|
|
|
If you have the worker processes managed by Supervisor, you can use the `supervisorctl` command-line tool to restart them:
|
|
|
|
```bash
|
|
supervisorctl restart group-name:*
|
|
```
|
|
|
|
>{notice} A more detailed [guide](#keeping-the-workers-running) on configuring Supervisor is included.
|
|
|
|
### Restarting Horizon
|
|
|
|
Similar to restarting regular worker processes, you can signal Horizon's master supervisor to terminate all worker processes by using the following command:
|
|
|
|
```bash
|
|
php artisan horizon:terminate
|
|
```
|
|
|
|
But in order to ensure your jobs won't be interrupted, you need to make sure of the following:
|
|
|
|
1. Your Horizon supervisors' `timeout` value is greater than the number of seconds consumed by the longest-running job.
|
|
2. Your job-specific `timeout` is shorter than the timeout value of the Horizon supervisor.
|
|
3. If you're using the Supervisor process manager to monitor the Horizon process, make sure the value of `stopwaitsecs` is greater than the number of seconds consumed by the longest-running job.
|
|
|
|
With this correctly configured, Supervisor will wait for the Horizon process to terminate and won't force-terminate it after `stopwaitsecs` passes.
|
|
|
|
Horizon supervisors will also wait for the longest job to finish running and won't force-terminate after the timeout value passes.
|
|
|
|
### Dealing With Migrations
|
|
|
|
When you send a restart signal to the workers, some of them may not restart right away; they'll wait for a job in hand to be processed before exiting.
|
|
|
|
If you are deploying new code along with migrations that'll change the database schema, workers that are still using the old code may fail in the middle of running their last job due to those changes; old code working with the new database schema!
|
|
|
|
To prevent this from happening, you'll need to signal the workers to exit and then wait for them. Only when all workers exit gracefully you can start your deployment.
|
|
|
|
To signal the workers to exit, you'll need to use `supervisorctl stop` in your deployment script. This command will block the execution of the script until all workers are shutdown:
|
|
|
|
```bash
|
|
sudo supervisorctl stop group-name:*
|
|
|
|
cd /home/forge/mysite.com
|
|
# ...
|
|
|
|
$FORGE_PHP artisan migrate --force
|
|
|
|
sudo supervisorctl start group-name:*
|
|
```
|
|
|
|
>{warning} Make sure the system user running the deployment can run the `supervisorctl` command as `sudo`.
|
|
|
|
Now, your workers will be signaled by Supervisor to stop after processing any jobs in hand. After all workers exit, the deployment script will continue as normal; migrations will run, and finally, the workers will be started again.
|
|
|
|
However, you should know that `supervisorctl stop` may take time to execute depending on how many workers you have and if any long-running job is being processed.
|
|
|
|
You don't want to stop the workers in this way if you don't have migrations that change the schema. So, I recommend that you don't include `supervisorctl stop` in your deployment script by default. Only include it when you know you're deploying a migration that will change the schema and cause workers running on old code to start throwing exceptions.
|
|
|
|
You can also manually run `supervisorctl stop`, wait for the command to execute, start the deployment, and finally run `supervisorctl start` after your code deploys.
|