What does the number of maximum containers per node mean in Hadoop?

1
votes

In Hadoop MapReduce Tutorial, I saw the words

The right number of reduces seems to be 0.95 or 1.75 multiplied by (< no. of nodes > * < no. of maximum containers per node >).

But I don’t know what no. of maximum containers per node is and how to set it. And I want to know how to set the number of reducers per node. I haven’t get any clues in docments yet.

1
hadoopmapreducehadoop-yarn
stackoverflow.com/questions/7247059/… this link has the answer you are looking for - kartheek
sorry, the appearance of original question was wrong. I re-edited it. - Ian Y

1 Answers

0
votes

A YARN NodeManager is started with a maximum JVM size

A YARN container has its own, configurable size.

Divide your smallest NodeManager by the YARN Container size for a job, and this will be the max containers for a node