Concurrency Limit across endpoints in MassTransit (C#)

1
votes

Mass Transit 3.5, RabbitMq

I have a bus connected service that listens on number of queues.

  • Q1
  • Q2
  • Q3
  • Q4 etc

All queues are accepting same message type, and all of them are using a competing consumer pattern, where there are multiple services on different machines listening on these same queues.

What I am after is setting a concurrency limit on all these queues together, per single process.

I have a limited resource (number of available licenses), and need to limit concurrent requests to number of available licenses per machine.

So machine A may have 4, machine B may have 10, etc.

I do not want machine A to consume any message from all these queues if I already have 4 consumers processing messages across all queues, it should let others (machine B, C, etc) to consume it if they have available resources.

My issue is that using

    sbc.UseConcurrencyLimit(4)

inside

    var bus = MassTransit.Bus.Factory.CreateUsingRabbitMq

is setting concurrency limit per queue, so if I have 4 of them, they are all added up.

What I need is for all of the queues to cumulatively work up to that concurrency limit, but not exceed it.

Is there a built-in way in MassTransit to achieve this goal?

1
c#concurrencyrabbitmqmasstransit

1 Answers

0
votes

There isn't a built-in way to limit concurrency across multiple services. That would require using some type of global resource manager, and it just isn't something that MassTransit supports out of the box.