Unicorns enter a restart loop on Heroku

Question

I have a rails application deployed to Heroku Celadon cedar stack using Unicorn (4.5.0) with the following unicorn.rb file:

worker_processes 2 # amount of unicorn workers to spin up
timeout 30         # restarts workers that hang for 30 seconds
check_client_connection true

At seemingly random times without any noticable change in services it uses(including DB) unicorns will enter a restart loop. They will keep restarting with the following typical error:

ERROR -- : worker=0 PID:935 timeout (31s > 30s), killing

The problem is that is keeps restarting more often than every 30 seconds per unicorn worker. It stops when the underlying dyno gets restarted so I'm guessing it has something to do with the way unicorn master process and heroku interact.

Anyone else experiencing this or has any ideas as to what could be the cause?

Tom Pesman Tom Pesman · Accepted Answer · 2013-03-04T20:16:06

You should not use the check_client_connection true option.

According to Heroku Unicorn documentation you should use a configuration file like this:

# config/unicorn.rb
worker_processes 3
timeout 15
preload_app true

before_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn master intercepting TERM and sending myself QUIT instead'
    Process.kill 'QUIT', Process.pid
  end

  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.connection.disconnect!
end  

after_fork do |server, worker|

  Signal.trap 'TERM' do
    puts 'Unicorn worker intercepting TERM and doing nothing. Wait for master to send QUIT'
  end

  defined?(ActiveRecord::Base) and
    ActiveRecord::Base.establish_connection
end

Unicorns enter a restart loop on Heroku

1 Answers