Geek Answers Handbook

Hadoop - increasing map tasks in xml does not increase map tasks when executed

I added the following to conf/mapred-site.xml

<property>
  <name>mapred.tasktracker.map.tasks.maximum</name>
  <value>4</value>
</property>

<property>
  <name>mapred.tasktracker.reduce.tasks.maximum</name>
  <value>1</value>
</property>

But when I start the task, does it still work 2 maps(which is default one)? How can I make this number increase?

PS I use Ubuntu Quad corebox

thank

+5
source share
3 answers

Do you use a small amount of data? Perhaps your MapReduce job only works with one input splitting and therefore does not require more parameters. Instead, try working on hundreds of MB of data and see if you have the same problem.

, node, , . 20 , 5 , . 50 , 2 .

+4

mapred.tasktracker.map.tasks.maximum - , tasktracker . , mapred.map.tasks 4.

+2
<property>
  <name>mapred.tasktracker.map.tasks.maximum</name>
  <value>1</value>
  <final>true</final>
</property>

Try it.

0
source

More articles:


All Articles