3
votes

For testing purposes, I want to run multiple instances of Hazelcast on a single node. Hazelcast assumes that it is managing the entire node, and therefore it creates a large enough number of threads to fully (in fact, over) load all the cores. On my 8-core Linux node, Hazelcast creates 74 threads per instance, which quickly runs up against the user thread limit. (Yes, I know this can be changed with ulimit.)

From jconsole threads view:

  • hz.hzInstance_1_dev.scheduled
  • hz.hzInstance_1_dev.generic-operation.thread (x16)
  • hz.hzInstance_1_dev.partition-operation.thread (x16)
  • hz.hzInstance_1_dev.response
  • hz.hzInstance_1_dev.event (x5)
  • hz.hzInstance_1_dev.wait-notify
  • hz.hzInstance_1_dev.wan (x16)
  • hz.hzInstance_1_dev.cached.thread
  • hz.hzInstance_1_dev.migration
  • hz.hzInstance_1_dev.IO-thread-in (x3)
  • hz.hzInstance_1_dev.IO-thread-out (x3)
  • hz.hzInstance_1_dev.IO-thread-Acceptor
  • hz.hzInstance_1_dev.cached-thread (x8)
  • hz.hzInstance_1_dev.HealthMonitor

Some of the thread counts can be reduced by setting Hazelcast system properties:

  • generic-operation.thread = hazelcast.operation.generic.thread.count
  • partition-operation.thread = hazelcast.operation.thread.count
  • IO-thread = hazelcast.io.thread.count
  • event = hazelcast.event.thread.count

The default numbers of wan, generic-operation and partition-operation threads are all 2 x CPU core count. There doesn't appear to be any way to reduce the number of wan threads.

Is there a way to specify how many cores Hazelcast should use, rather than specifying generic-operation.thread and partition-operation.thread separately? Setting CPU affinity doesn't work, as Hazelcast still thinks it has access to all the cores.

Is there any way to change the number of WAN replication threads?

1

1 Answers

1
votes

For Hazelcast 3.4 by default the partition operation thread count will be 1x core-count.

There is no option to configure number of operation threads apart from these properties.

AFAIK it isn't possible to change the number of wan replication threads since it is hardcoded to: ExecutorConfig.DEFAULT_POOL_SIZE.

A thread that doesn't do anything, it very cheap. The OS scheduler doesn't schedule it; just needs to track data.