DELETE PERFECT Nodes

Question

In our YARN cluster, which is 80% full, we see that some of the thread nindeamers are marked as UNHEALTHY. after digging into the logs, I found it because the disk space is 90% full for the dir data. With the following error

2015-02-21 08:33:51,590 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Node hdp009.abc.com:8041 reported UNHEALTHY with details: 4/4 local-dirs are bad: /data3/yarn/nm,/data2/yarn/nm,/data4/yarn/nm,/data1/yarn/nm; 2015-02-21 08:33:51,590 INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: hdp009.abc.com:8041 Node Transitioned from RUNNING to UNHEALTHY 

I am trying to understand how node yarn labels are Unhealthy and is there a way to change the threshold?

thanks

Answer 1

try adding the yarn.nodemanager.disk-health-checker.max-disk-usage-per-disk-percent property to yarn-site.xml. This property indicates the maximum percentage of disk space used, after which the disk is marked as bad. Values ​​can vary from 0.0 to 100.0.

yarn-default.xml

strength for health status for example :.

 <?xml version="1.0"?> <configuration> <property> <name>yarn.nodemanager.disk-health-checker.min-healthy-disks</name> <value>0.0</value> </property> <property> <name>yarn.nodemanager.disk-health-checker.max-disk-utilization-per-disk-percentage</name> <value>100.0</value> </property> </configuration>