Geek Answers Handbook

MapReduce custom input format - cannot find constructor

I am writing a custom InputFormat for Hadoop 0.20.2 and am in a NoSuchMethodException that I cannot get rid of. I started with:

public class ConnectionInputFormat extends FileInputFormat<Text, Connection> { @Override public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException { return new ConnectionRecordReader(); } }

I got this error on startup:

 Exception in thread "main" java.lang.RuntimeException: java.lang.NoSuchMethodException: testingground.TestInputJob$ConnectionInputFormat.<init>() at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:115) at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:882) at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:779) at org.apache.hadoop.mapreduce.Job.submit(Job.java:432) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:447) at testingground.TestInputJob.run(TestInputJob.java:141) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79) at testingground.TestInputJob.main(TestInputJob.java:156) Caused by: java.lang.NoSuchMethodException: testingground.TestInputJob$ConnectionInputFormat.<init>() at java.lang.Class.getConstructor0(Class.java:2706) at java.lang.Class.getDeclaredConstructor(Class.java:1985) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:109) ... 8 more Java Result: 1

After running the error for the first time and researching the Internet, I thought that maybe I don't have a constructor with a null argument, so I added one:

 public class ConnectionInputFormat extends FileInputFormat<Text, Connection> { public ConnectionInputFormat() { System.out.println("NetflowInputFormat Constructor"); } @Override public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException { return new ConnectionRecordReader(); } }

This did not work either, so I added a second constructor that took up any number of objects:

 public class ConnectionInputFormat extends FileInputFormat<Text, Connection> { public ConnectionInputFormat() { System.out.println("NetflowInputFormat Constructor"); } public ConnectionInputFormat(Object... o) { System.out.println("NetflowInputFormat Constructor"); } @Override public RecordReader<Text, Connection> createRecordReader(InputSplit split, TaskAttemptContext context) throws IOException, InterruptedException { return new ConnectionRecordReader(); } }

Still getting the same error and still have not been able to find a solution.

Full power source: http://pastebin.com/2XyW5ZSS

+7
source share
1 answer

Your ConnectionInputFormat class must be static. Non-static nested classes mean 'this', added to each constructor. So your no-arg constructor actually has an invisible argument unless the class is declared static.

+7
source

More articles:


All Articles