I am using beanstalkd in a Laravel project to handle jobs on a queue. Beanstalkd is running locally. What I want to do is add one or more remote servers to handle some jobs when the queue gets bigger. I know that with Laravel I can send a job to a specific remote connection but in this way I don't know the load in each server prior to sending the job.
I was wondering if beanstalkd supports load balancing between servers and error handling when a remote job fails for example.
Thank you
Beanstalkd does't have features for load balancing.
You could setup a HAProxy on your balancer and signup multiple servers with beanstalkd installed. Then when you send jobs from Laravel code you send to the HAProxy, and HAProxy decides on which sub-server puts the job, as it knows the loading and if there is an incident with a sub system.
In the code you just need to change the IP. In your infrastructure you need to have balancer (HAProxy) that is setup with a pool of Beanstalkd servers.
We usually have 2 machines, and they are configured like this:
- Machine 1: HAProxy, Apache, MySQL, Laravel, Beanstalkd
- Machine 2: MySQL, Laravel, Beanstalkd
