Geek Answers Handbook

Differences between Hadoop-common, Hadoop-core and Hadoop client?

I am new to Hadoop and want to know what is the difference between Hadoop-common , Hadoop-core and Hadoop client

By the way, for this class, how do you know which artifact contains it in Maven ? For example, which one contains org.apache.hadoop.io.Text ?

+8
maven hadoop
source share
3 answers

To provide more information on the differences between the Hadoop-common, Hadoop-core, and Hadoop client, from a high perspective:

  • Hadoop-common refers to commonly used utilities and libraries that support Hadoop modules.
  • Hadoop-core is the same as Hadoop-common; It was renamed Hadoop-common in July 2009, for https://hadoop.apache.org/ .
  • The Hadoop client refers to the client libraries used to communicate with common Hadoop components (HDFS, MapReduce, YARN), including, but not limited to, logs and codecs, for example.

Generally speaking, for developers who create applications that send YARN, run an MR job, or access files from HDFS, use the Hadoop-client libraries.

+6
source share

To create a Hadoop map abbreviation application, you only need the hadoop client dependency. (Use the new API). Dependencies such as hadoop-hdfs, hasoop-common, hadoop-clientapp, hasoop-yarn-api have been removed from this.

+1
source share

From techopedia

Hadoop Common refers to a collection of common utilities and libraries that support other Hadoop modules. This is an important part or module of the Apache Hadoop Framework, along with the Hadoop Distributed File System (HDFS), Hadoop YARN, and Hadoop MapReduce.

Like all other modules, Hadoop Common assumes that hardware failures are common and that they should be handled automatically using the Hadoop Framework.

Hadoop Common also known as Hadoop Core .

Hadoop Client libraries help you load data into the cluster, passes Map Shorten tasks that describe how this data should be processed, and then retrieve or view the results of the task after it is completed. Take a look at the article

This Apache link contains a list of dependencies of the Hadoop Client library.

0
source share

More articles:


All Articles