Geek Answers Handbook

JobConf v / s Configuration for Hadoop 1.0.4

Hi, I am new to Hadoop and this is FileSystem. I saw two different WordCount examples using JobConf and Configuration . What is the difference in them.

I learned that JobConf was part of the old org.apache.hadoop.mapred package (which was deprecated at 0.20.x), but Configuration is part of the new org.apache.hadoop.mapreduce package. But now in version 1.0.4 it is deprecated.

Currently, we have two ways to perform map reduction jobs in java, one using (extension) classes in the org.apache.hadoop.mapreduce package, and the other through implementing classes in the org.apache.hadoop.mapred package.

I want to know:

  • What is the difference between mapred and mapreduce package structure and why mapred not outdated?

  • Which approach is better to use v1.0.4 to use and why? JobConf or Configuration ?

  • What is better for v1.0.4? mapred or mapreduce ?

+7
source share
1 answer

If you look in the releases page , you will see that 1.0.4 corresponds to something around 0.20.20x

To give some context, here is what was discussed on the mailing list :

 The "old" MapReduce API in org.apache.hadoop.mapred was deprecated in the 0.20 release series when the "new" (Context Objects) MapReduce API was added in org.apache.hadoop.mapreduce. Unfortunately, the new API was not complete in 0.20 and most users stayed with the old API. This has led to the confusing situation where the old API is generally recommended, even though it is deprecated.

So, as you can see, this is basically a retro-compatibility issue.

So, on the bottom line, if you are starting the application from 1.0.4 now, you should use mapreduce , not mapred , as this is the preferred way now, but you can use the old mapred if you have outdated applications. This means that you must use Configuration .

As for the difference between mapred and mapreduce , as explained in the excerpt above, this is mainly due to the introduction of Context objects, but there are many other changes and new classes that are not available in the old mapred .

+6
source

More articles:


All Articles