How to save the output of fuzzy gears in HDFS

Question

In my application, the reducer saves all part files in HDFS, but I want only the reducer to write part files whose size is not 0 bytes. Please let me know how to identify it.

Answer 1

It is possible - see the documentation section on "Lazy Output":

http://hadoop.apache.org/mapreduce/docs/current/mapred_tutorial.html#Lazy+Output+Creation

import org.apache.hadoop.mapreduce.lib.output.LazyOutputFormat; LazyOutputFormat.setOutputFormatClass(job, TextOutputFormat.class); 

Answer 2

If you are using the old API, you can use the NullOutputFormat class:

 import org.apache.hadoop.mapred.lib.NullOutputFormat; conf.setOutputFormat(NullOutputFormat.class);