I am using Sidekiq with my Rails app and have about 30 workers that kick off in random orders depending on the tasks clicked on by the user in the rails app. Sometimes, a sidekiq job fails and I'll have to shut down sidekiq and investigate the error manually, which consequently interrupts all workers (initiated by that user and any other user)
I just implemented a middleware so that I can finally call a method every time a sidekiq job fails; however, now I'm wondering if there's a way that I could re-run that job in a different queue (let's say "retry_queue").
Right now, my sidekiq workers start off like this:
class TestingWorker
include Sidekiq::Worker
sidekiq_options queue: Rails.env.to_sym
This basically uses whatever queue the environment is running in (e.g. development or production). However, I'm not sure how to accomplish my goal given this or any other setup.
So my questions are:
- How can I redirect a failed sidekiq job to another queue from this middleware method that captures failed sidekiq workers?
- After redirecting it to another queue, how can I re-design the sidekiq job to understand this?
Ideally, if I could somehow change this:
sidekiq_options queue: Rails.env.to_sym
to something like this:
sidekiq_options queue: [something here].include? "retry_queue" ? :retry_queue : Rails.env.to_sym
Any thoughts or suggestions would be greatly appreciated.
