Start and stopping workers in gearman php

Question

I've successfully configured and used gearman and its pecl php extension. I'm using it to execute a long process concerning long sql queries in a background. I'm using Yii btw, if that detail helps.

Here's how I use it :

public function actionProcessWithGearman(){

$output = shell_exec('gnome-terminal -e "php workers/worker.php" > /dev/null 2>/dev/null &');


$client = new GearmanClient();
$client->addServer();
$result = $client->doBackground('executeJob',//parameters);

}

Some details:

If you notice I run a gnome-terminal first, so that I can see the process rather than going directly with the php command, I also added /dev/null so that it will no longer wait for a response. And then the worker is woken up and runs the job.

Problem:

My problem arises when this action is executed several times or executed by several users in different clients, and as a result, multiple terminals running worker.php are being instantiated.

How do I have only one worker? and even if I can have several workers for several users in the different clients, how do I close a worker everytime the task is finished?

Answer 1

You can try adding returnCode() and jobStatus().

A sample here - http://php.net/manual/en/gearmanclient.dobackground.php

Answer 2

If you have a queue system in place the workers should always be running in the background and be ready to catch new jobs.

I think there is a misunderstanding on the queue system logic. You should just add a new job that will be processed by the worker, and not start the worker if you want a job to be done.

Gearman usually goes well with supervisor. Supervisor will control the worker and make sure it will be always available. If you need more than one job to be running at the same time then you can always add a new worker.

Just look for gearman + supervisor + php there are a lot of articles explaining how to set that up.

Answer 3

Letting supervisord taking care of the php processes launch and monitoring is the way to go.

This approach have a few pluses: for instance you can let supervisord start workers automatically, even multiple instances of it. And you can even control supervisord processes through xmlrpc, if you prefer to manage your workers from your own web interface (http://supervisord.org/api.html)

Sample supervisord conf

[program:worker_development]
process_name=worker_%(process_num)s
command=php worker.php
directory=/var/ww/myproject
autorestart=true
user=ubuntu
redirect_stderr=false
numprocs=2
startretries=5