How to manage different queue for each tenant in Multi-tenant SaaS application?

2
votes

We are running a multi-tenant SaaS application for the reward program.

Currently, we are managing each tenant's queue separately in the database.

Each tenant can have a notification, reward, bulk_operations queues.

Currently, we are managing each queue in a single queue of that tenant.

We know that this is not the best approach and best practice for doing it.

We are planning to scale our application in and we need to set up queues and queue worker in the best possible way.

Any suggestions would be great for that. We are open to using any services for queue like SQS, Redis, etc. We need a separate queue for each tenant.

[Technicle details]

Server:- AWS ec2

Language:- PHP

Framework:- Laravel 5.6

Database:- RDS Mysql

[Requirements in terms of Throughput/Scalability]

  1. We have some tenant who has 100K jobs in a week or a day that might be increasing in the future as they are e-commerce store.
  2. We have a daily minimum of 5-6 new tenants. So we need to manage their queues as well.
  3. Each tenant's queue should be sperate so that one tenant doesn't need to depend on other tenant's process to be finished first. (Each tenant's process should be managed separately).

[For more details on current structure]

Please visit this question for a more detailed understanding of the current structure here -> Click Here

1
laravelamazon-web-servicesqueuemulti-tenantsaas
What are your requirements in terms of throughput, scalability, etc.? – Namoshek
In that question, we just ask about Laravel database queue management. @dparoli – Milind Patel
@Namoshek I have updated questions with details as you asked. Please review it. – Milind Patel
@dparoli Added (y). – Milind Patel

1 Answers

1
votes

Redis works fine for starters. Easy to configure and install. You should also try laravel horizon. You can easily configure queues on horizon at config/horizon.php from there you can create separate notification, reward, bulk_operations connections you just need to specify it on your dispatch. 

ProcessPodcast::dispatch($podcast)->onQueue('processing');

you can check the official documentations of laravel horizon here https://laravel.com/docs/5.6/horizon

and for the queues and connections here https://laravel.com/docs/5.6/queues#customizing-the-queue-and-connection