Service Fabric Reliable Collections Performance

Question

I need to store a lot of data in Reliable Dictionaries on Service Fabric. We are implementing an event store as a number of Reliable Dictionaries, so every event emitted by the domain ends up in the store. I would like to know the difference in performance in the two following scenarios:

• use one (very large) Reliable Dictionary to store all events for a certain aggregate type : this results in a small number of dictionaries, each containing millions of events
• use a small Reliable Dictionary to store the events of a single aggregate instance : this results in a LOT of small dictionaries (think millions) each containing a few events

In light of the replication of state, and read and write performance, what would be the most efficient way forward?

Answer 1

Sounds like you should use Stateful Actors, you can have millions of actors holding data.

If you need to read a lot of summary (aggregated) information from all of your actors, pls see https://github.com/Azure-Samples/service-fabric-dotnet-data-aggregation/blob/master/README.md

Here are my thoughts if you want to go with stateful services: For the first scenario you will have to use partitioning, for the second one you should build multiple data services, so that your data is distributed among the nodes.

The second scenario has the advantage of quicker access to its data, but requires an extra catalog service to hold the names of the event aggregates, so that you can lookup the correct target dictionary.

Replication should not be different in the two scenarios.