Difference between local PIG mode and mapreduce mode

Question

What is the actual difference between running PIG scripts locally and on mapreduce? I understand that mapreduce mode is when you run it in a cluster with hdf installed. Does this mean that the local mode does not require HDFS, and therefore even mapreduce jobs do not start? What is the difference, and when are you different?

Answer 1

Local mode will create a simulated mapreduce job that runs from a local file on disk. Theoretically equivalent to MapReduce, but this is not the "real" work of mr. You cannot distinguish the user from the perspective.

Local mode is great for development.

Answer 2

Local mode: all scripts run on the same machine without using Hadoop MapReduce and HDFS. This can be useful for developing and testing Pig logic. If you use a small data set for the developer or check your code, then local mode can be faster than through the MapReduce infrastructure.

Local mode does not require Hadoop. When launched in local mode, the Pig program runs in the context of the local Java virtual machine, and data is accessed through the local file system on the same machine. Local mode is a local MapReduce simulation in the Hadoops LocalJobRunner class.

MapReduce mode (also known as Hadoop mode): The pig runs on a Hadoop cluster. In this case, Pig Script is converted to a series of MapReduce jobs, which are then run on the Hadoop cluster.

If you have a terabyte of data for which you want to perform operations, and you want to interactively develop a program, you can slow down significantly soon, and you can start building up your storage. Local mode allows you to work with a subset of your data in a more interactive way so that you can figure out the logic (and develop errors) of your Pig program.

Once you have everything you need and your operations run smoothly, you can run Script for a complete set of data using the MapReduce mode.