Apache Ignite - is there a way to avoid hot spots nodes in collocated computations

1
votes

i have an Ignite cluster configuration, where a given data set saved in Ignite-cache and computations are collocated accordingly, it reaches a point where adding more nodes does not improve performance.

I noticed that this is because some of the nodes are hot-spot and computes more than others because of collocation and desired data in hot spot. cache data structure cannot be changed.

I wonder if there is a configuration in Ignite where i can scale the hot-spot nodes - meaning fully replicate them. then, compute tasks they get is load balanced between the replications.

is that possible? any idea how to configure?

1
scaleignite
It sounds like you need to help the partitioning algorithm to spread the hot data entries across the cluster evenly. Do you know what entries are hot? If yes, then you can implement your own AffinityFunction. Check this discussion thread for initial guidance: apache-ignite-users.70518.x6.nabble.com/… – dmagda
The Affinity is already optimal. there are some keys that are more requested than others, but they are fully spread. think that there are much less hot keys than nodes to scale out – chen

1 Answers

0
votes

If you know which keys are hot, you can put them into the smaller REPLICATED cache and execute computes on any node instead of affinity node.