Sharing Azure Service Fabric Reliable Queues Between Services

Question

I'm diving into Service Fabric (from the Cloud Services world) and am hitting a few speed bumps with how ReliableQueues work.

Let's say I have 2 stateful services StatefulService1 and StatefulService2.

If I need to have StatefulService1 send a message in a queue that StatefulService2 will pick up and read am I able to use ReliableQueues or are ReliableQueues isolated within the service they are created in?

If that is the case then what is the purpose of having ReliableQueues? The usual pattern behind them is for another process to act on the messages. I understand why isolating a Dictionary to a service would make sense, but not a queue...

Is my best option to rely on a traditional approach to send this message such as a Storage Queue or does ServiceFabric offer a solution for passing message queues between services?

UPDATE

Just want to clarify that I did attempt to dequeue a message created in StatefulService1 from within StatefulService2 and it came up empty. Dequeing from within StatefulService1 worked fine as expected.

Answer 1

Reliable Collections (queue and dictionary) are not intended for communication. With queues, it's a 2PC, so only one process can access it at any point in time. Note that when you use stateful services with partitions, to access the data both service instances have to be on the same partition. Different partitions cannot access the same data.

Relying on either traditional methods or implementing your own communication listener is the way to go. With the traditional way - keep in mind that you'll need to decide if you want to partition your queues just like your services are or not.

Answer 2

Reliable Collections are in memory data structures that are not intended for inter-service communications. If you would like to establish a communication channel between StatefulService1 and StatefulService2, you have the following options:

1. Use Communication Listeners. You can have custom listeners for the protocol of your choice, including HTTP, WCF or your custom protocol. You can read more about it in this section. For example, StatefulService2 can open up an HTTP endpoint that StatefulService1 can POST/GET to.

2. Use an external queuing system, like Servicebus, EventHub or Kafka, where StatefulService1 can post events to. StatefulService2 can be consumer service that consumes events from the queue and processes it.

Answer 3

I don't see why a service can't host a reliable collection/queue, and other services can access it via one of three transports: Remoting, WCF and HTTP.
Obviously, the reliable service will have to expose the collection/queue via an API or implement an IService interface

https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-connect-and-communicate-with-services

Answer 4

You have to add a fault-handling retry pattern to your calling code, see https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-reliable-services-communication, in this case you don't need a queue to hold data for your in between service calls.