Sidekiq jobs stuck in queue on Heroku

Question

I have a Sidekiq worker functioning well locally, but when deployed to Heroku the jobs get stuck in the queue. I am using Redis-to-go nano and have it up and running, and I have scaled the worker to 1 on Heroku and can see that it is up. I am just using the default queue -- nothing custom or fancy. Here is my code:

config/unicorn.rb:

Sidekiq.configure_client do |config|
  config.redis = { size: 1, namespace: 'sidekiq' }
end

config/initializers/redis.rb

uri = URI.parse(ENV["REDISTOGO_URL"] || "redis://localhost:6379")
REDIS = Redis.new(:url => ENV['REDISTOGO_URL'])

Procfile

web: bundle exec unicorn -p $PORT -c ./config/unicorn.rb
worker: bundle exec sidekiq -c 5 -v -q default

I can see the job in the queue but it is not processing like it does locally. Any advice is much appreciated - thanks!

Try running heroku run console and run your job synchronously SomeWorker.new.perform(some_arg), it might be throwing an error and get rescheduled. — Ollie
Hi, thanks for the quick response. Running in the heroku console performed the actions I expected (i.e. it generated a new user with the correct parameters); however, I can't see any evidence that it worked in the Sidekiq web UI -- that is, it was neither enqueued not processed. Any thoughts? — Andy Weiss
Once I got it to work from the heroku console, just out of curiosity I tried to run the job synchronously from within my app.... But the jobs still get stuck in the queue even though I'm passing the exact same arguments. — Andy Weiss

Andy Weiss Andy Weiss · Accepted Answer · 2014-10-10T15:38:03

Not sure exactly what the problem was, but following this tutorial with some modifications worked out for me: http://manuelvanrijn.nl/blog/2012/11/13/sidekiq-on-heroku-with-redistogo-nano/

In short, I moved the configuration into a sidekiq.rb initializer, and deleted all of the url and namespace information.

require 'sidekiq'

Sidekiq.configure_client do |config|
config.redis = { :size => 1 }
end

Sidekiq.configure_server do |config|
config.redis = { :size => 4 }
end

The tutorial link I referenced has a handy calculator to detemrine the correct size values. Still not sure whether that's what was tripping me up or some version of the namespace conflict alluded to in Mark's answer.

Also, I didn't use the sidekiq.yml portion of the tutorial, because the sidekiq wiki says newer versions of rails don't like it. Instead, I set concurrency to 2 in the bundle exec command of the Procfile, like this:

worker: bundle exec sidekiq -c 2

Hope this is helpful to anyone who has a similar issue in the future! Thanks to all who tried to help.

Sidekiq jobs stuck in queue on Heroku

3 Answers