Ruby / Resque / Redis: How to set up workers on different machines?

Question

I've been looking into resque for distributed background jobs. I managed to get everything working on one machine through the demo app. But I don't know how to set everything up so the workers run on different machines. I read that it's just necessary that the workers have a connection to the redis server. But I still have no idea where to start. Is there a step-by-step instruction somewhere? I didn't find any documentation about this.

Or can someone please explain it to me?

I don't use rails btw, I'm using Sinatra.

Thx

Answer 1

Yes you are correct If all your worker can access the redis you can hook them up to distribute them on different machine

For that you need worker to be connected to redis (I assume are available to all worker)

Now in Resque to connect to that remote redis all it has to do is like this

Resque.redis = "redis://[your host]:[your port]"

you can also see the same it mention over here

Perhaps define it in your config/application.rb

Hope this help

Answer 2

The Redis security guide suggests that it is unsafe to expose your Redis server to the public internet. So, your safest bet is probably to create an SSH tunnel from each worker machine to the Redis server. You can find instructions here:

https://briandamaged.org/blog/?p=1675

Once you have secured your Redis server and established an SSH tunnel, your workers can connect to the server by accessing a port on localhost. For instance:

# Port 2000 is a tunnel to the Redis server
Resque.redis = "redis://localhost:2000"