Multiple resque workers mode creating extra processes

Question

I need to start 4 resque workers so i used following command

bundle exec rake environment resque:workers RAILS_ENV=production COUNT=4 QUEUE=* VERBOSE=1 PIDFILE=tmp/pids/resque_worker.pid  >> log/resque_worker_QUEUE.log

But going to web interface, it was actually starting 8 workers. There were two parent processes with 4 child processes each. Following is tree view of the processess:

ruby /code_base/bundle/ruby/1.9.1/bin/rake environment resque:workers RAILS_ENV=production COUNT=4 QUEUE=* VERBOSE=1 PIDFILE=tmp/pids/resque_worker.pid
 \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
     \_ [ruby] 
ruby /code_base/bundle/ruby/1.9.1/bin/rake environment resque:workers RAILS_ENV=production COUNT=4 QUEUE=* VERBOSE=1 PIDFILE=tmp/pids/resque_worker.pid
 \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
 |   \_ [ruby] 
 \_ resque-1.15.0: Waiting for *                                                                        
     \_ [ruby]

Couldn't figure out what is causing extra process to start?

Winfield Winfield · Accepted Answer · 2011-09-01T15:36:06

You don't want to use the COUNT=n option in production, as it runs each worker in a thread instead of a separate process - which is much less stable.

Official Resque docs:

Running Multiple Workers

At GitHub we use god to start and stop multiple workers. A sample god configuration file is included under examples/god. We recommend this method.

If you'd like to run multiple workers in development mode, you can do so using the resque:workers rake task:

$ COUNT=5 QUEUE=* rake resque:workers
This will spawn five Resque workers, each in its own process. Hitting ctrl-c should be sufficient to stop them all.

Here's the example God monitoring/configuration file that ships with Resque to run multiple processes, and here's an example for monit.

Multiple resque workers mode creating extra processes

1 Answers