Geek Answers Handbook

How to increase the number of containers in nodemanager in YARN

A node in my YARN cluster has 64 GB of memory and 24 cores. I set the following properties in yarn-site.xml:

<property> <name>yarn.nodemanager.resource.memory-mb</name> <value>32768</value> </property> <property> <name>yarn.nodemanager.resource.cpu-vcores</name> <value>16</value> </property>

But I found that nodemanager in node has only 7 containers. What other properties do I need to set?

+7
hadoop yarn
source share
2 answers

You need to tell YARN how to break the memory into containers, for example, if you install the memory on a 2 GB container, you will get 16 containers.

 <name>yarn.scheduler.minimum-allocation-mb</name> <value>2048</value>
+13
source share

try something like:

  <property> <name>mapreduce.map.memory.mb</name> <value>1024</value> </property> <property> <name>mapreduce.reduce.memory.mb</name> <value>2048</value> </property>
0
source share

More articles:


All Articles