Scaling multiple gearman workers

Question

I created 500 gearman workers on server that have 32 cores and 64GM RAM. The server working well with 60-70% CPU, 22 load avg. i created it with supervisor.

What is the best practice for this amount of workers, can you advise to other solution that is not supervisor? what about gearman manager?

The worker process a job between 10-100MS - on avg the queue is always 90%.

Is there any problem with the current solution? You're starting the workers with supervisord, did i get that right? — tillz
Yes, I started the workers with a supervisor, but I see that the supervisor is not running and doesn't do auto restart to workers that "die". When I start only with 200 workers (not 500) I don't have this problem, meaning the supervisor restarts the workers automatically. — vladi03

vladi03 vladi03 · Accepted Answer · 2016-04-13T13:43:14

1

votes

Want to update, when i upgrated the version of supervisor to 3.2 its solved my problem.

Scaling multiple gearman workers

1 Answers