Geek Answers Handbook

Hadoop EOF exception after map step

I run Hadoop in pseudo-distributed mode. My new MR jobs raise an EOF after completing the Map step in the DataInputStream.readFully () method. The input / output formats are the same - SequenceFile. Similar work is performed normally, but a new one is not.

Stacktrace:

java.io.EOFException at java.io.DataInputStream.readFully(DataInputStream.java:180) at java.io.DataInputStream.readFully(DataInputStream.java:152) at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1520) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1492) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1479) at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1474) at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.initialize(SequenceFileRecordReader.java:50) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:451) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:646) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323) at org.apache.hadoop.mapred.Child$4.run(Child.java:266) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInf

I fixed this problem, it was caused by several damaged sequence files in the HDFS input folder

0
eof hadoop
source share

No one has answered this question yet.

See similar questions:

8
Loading a bush into a partitioned table

or similar:

2
Mapreduce combfileinputformat java.lang.reflect.InvocationTargetException, while two jobs have access to the same data
2
Hadoop-streaming: PYTHONPATH does not work when mapper starts
one
SplitFile gives casting error
one
KiteSDK MapReduce: EOF exception when loading a parquet file
one
Msgstr "Type of inconsistency in the key from the map: expected org.apache.hadoop.io.IntWritable, received by org.apache.hadoop.io.LongWritable" -Every thing looks right
0
Map mismatch type
0
PDF input format for Mapreduce Hadoop
0
Hadoop - unable to catch ioexception when pasting in mysql
0
Passing arguments to write reads in mapreduce hadoop
0
Error loading .gz files in Pig Script

More articles:


All Articles