Load balancing a service fabric StatelessService with ServicePartitionClient

Question

I'm implementing an API gateway on a service fabric cluster where the API gateway service is the public endpoint that routes external HTTP requests into a set of worker services running in the cluster. For inter-service communication between the gateway and the internal services, we're using the ServicePartitionClient.

I've found that when using the ServicePartitionClient to resolve service addresses and communicate with an stateless service, it will pick a single instance of the stateless service and only communicate with that one instance with every communication attempt. In my case, I have multiple instance of my stateless service running and would like to distribute load across them (e.g. round robin). Is there a way to do this using ServicePartitionClient that doesn't require a hit to the NamingService service every time (which is too expensive for our use case)?

Answer 1

Create a new instance of ServicePartitionClient for each request.

ServicePartitionClient is a relatively light-weight data structure that just holds some metadata about a communication channel. The actual connection management, pooling, and resolved-name caching happens in the layers underneath:

• FabricClient caches resolved endpoints so you don't hit the Naming Service every time you ask for an endpoint.
• ServicePartitionResolver wraps FabricClient with a basic retry-loop based on common connection exceptions we know about.
• CommunicationClientFactoryBase, if you're using it, holds on to a ServicePartitionResolver instance for you and caches connections.

So just make sure you are reusing your CommunicationClientFactory. If you are using CommunicationClientFactoryBase and you aren't passing in your own ServicePartitionResolver, it uses the singleton by default.