Heroku Procfile with multiple workers

Question

I have an app with 1 web dyno, 7 background queues, and 1 clock. I want to use the Heroku Standard 1x plan as it includes unlimited background workers (and the number of queues might increase further in the future). When I deploy my app, the procfile doesn't appear to provision the way I expected. Three dynos are shown, and it looks like I have to purchase an additional Standard 1x dyno for each worker/clock.

Procfile:

web: flask db upgrade; gunicorn webapp:app
worker: rq worker -u $REDIS_URL high default low
worker: rq worker -u $REDIS_URL r1
worker: rq worker -u $REDIS_URL r2
worker: rq worker -u $REDIS_URL bg1
worker: rq worker -u $REDIS_URL bg2
worker: rq worker -u $REDIS_URL bg3
worker: rq worker -u $REDIS_URL bg4
clock: python clock.py

Heroku shows 3 dynos:

1x web (enabled, scale:1, Standard 1x)
1x worker "bg4" (scale:0)
1x clock (scale:0)

How should I structure the Procfile to take advantage of the background workers included in the Standard 1x dyno?

dduffy dduffy · Accepted Answer · 2021-05-27T12:51:03

Firstly, there was an error in how I structured the procfile. Each "worker" should be named differently. I thought the term "worker" held some special meaning for Heroku provisioning; it doesn't. So Procfile should look something like this:

web: flask db upgrade; gunicorn webapp:app
worker1: rq worker -u $REDIS_URL high default low
worker2: rq worker -u $REDIS_URL r1
worker3: rq worker -u $REDIS_URL r2
worker4: rq worker -u $REDIS_URL bg1
foo_worker5: rq worker -u $REDIS_URL bg2
more_worker6: rq worker -u $REDIS_URL bg3
another_worker7: rq worker -u $REDIS_URL bg4
clock: python clock.py

Secondly, the answer to my actual question (thanks to Heroku support team) is that a seperate Dyno is required for each additional worker process. I feel how it's phrased on the Heroku pricing page is a little missleading.

The support team were able to clarify:

In this context, "unlimited background workers" means you can run as many worker processes as you need to. On the lower tiers(free and hobby), the number of worker processes that can be used is limited.

However, Heroku is built on a 1-process-per-dyno model, which means every worker needs to run on its own dyno. So you will be charged for 1 dyno for each worker process you use. To say it a different way, it's not "unlimited worker processes for $25/mon", but instead it's "run unlimited worker processes at $25/mon each".

Basically, the Free tier can have up to 2 dynos, Hobby can have up to 10 and the Production tiers can have as many as you want.

Heroku Procfile with multiple workers

1 Answers