Laravel Queues - how to assign specific number of workers to different queues

Question

I've set up Laravel (5.5) queuing with Beanstalkd, and I'm using Supervisor. Laravel docs mention the following:

By pushing jobs to different queues, you may "categorize" your queued jobs and even prioritize how many workers you assign to various queues.

but it's never mentioned how to do that. If I have 2 queues on my beanstalkd connection, and I do the following:

php artisan queue:work beanstalkd --queue=high,low

That will always process ALL of the jobs from high before it processes anything from low, if I'm understanding the docs correctly:

To start a worker that verifies that all of the high queue jobs are processed before continuing to any jobs on the low queue, pass a comma-delimited list of queue names to the work command

Let's say my queues are called apprequests and emails. How would I go about adding 1 dedicated worker for emails queue which won't care about the apprequests queue, and then 4 workers for the apprequests queue?

Here's my Supervisor laravel_worker.conf config:

[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/my_app/artisan queue:work beanstalkd --queue=apprequests --timeout=330
autostart=true
autorestart=true
user=root
numprocs=4
redirect_stderr=true
stdout_logfile=/var/www/my_app/worker.log

Am I right to assume that I can just add another Supevisor config, for example laravel_email_worker.conf, with this command - command=php /var/www/my_app/artisan queue:work beanstalkd --queue=emails --timeout=30 and set numprocs to 1, and that should dedicate 1 worker to my emails queue which will process jobs from that queue regardless of the apprequests queue, or am I going all wrong about this?

Marcin Nabiałek Marcin Nabiałek · Accepted Answer · 2017-11-15T21:25:10

Yes, you can do as you wrote. You can set up one process that will look at email queue jobs and 4 processed that will look at apprequests queue.

But you might be also interested in Laravel Horizon that uses Redis as queue driver and can be set up to auto balance queue to increase number of workers for certain queues depending on current load.

Laravel Queues - how to assign specific number of workers to different queues

1 Answers