Running JUnit tests with Hadoop 2.2 on Windows 7

I am trying to run a simple JUnit test with Hadoop 2.2. on my Windows 7 machine. I have an example test available at http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/ , so I use ClusterMapReduceTestCase as the base class to run MiniDFSCluster .

When I run the test, I get (in the console):

2013-11-15 11: 59: 24,636 ERROR [main] util.Shell (Shell.java:getWinUtilsPath (303)) - Could not find winutils binary in hadoop binary path java.io.IOException: Could not find null executable \ bin \ winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath (Shell.java:278) at org.apache.hadoop.util.Shell.getWinUtilsPath (Shell.javahaps00) on org.apache.hadoop.util.Shell. (Shell.java:293) at org.apache.hadoop.util.StringUtils. (StringUtils.java:76) (...)

and at the output of JUnit:

java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO $ Windows.access0 (Ljava / lang / String; I) Z on org.apache.hadoop.io.nativeio.NativeIO $ Windows.access0 (native method ) on org.apache.hadoop.io.nativeio.NativeIO $ Windows.access (NativeIO.java:435) on org.apache.hadoop.fs.FileUtil.canWrite (FileUtil.java:996) at org.apache.hadoop. hdfs.server.common.Storage $ StorageDirectory.analyzeStorage (Storage.java:451) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs (FSImage.java:282) at org.apache.hadoop.hdfs. server.namenode.FSImage.recoverTransitionRead (FSImage.java:200) (...)

I read the topics: I could not find the winutils binary in the hadoop binary path and Starting Apache Hadoop 2.1.0 on Windows I also examined the possible solutions available at http://www.srccodes.com/p/article/38/build-install -configure-run-apache-hadoop-2.2.0-microsoft-windows-os and http://www.srccodes.com/p/article/39/error-util-shell-failed-locate-winutils-binary-hadoop -binary-path

Unfortunately, this does not work with JUnit tests. If I write junit tests, I want them to be independent of the development environment (so that this can work without problems on every machine for developers). Therefore, I use maven, and I do not want to create the necessary libraries myself and put them in the Hadoop bin folder, etc.

Any suggestions what would be the best solution in this case?

+7
java windows junit hadoop
source share

No one has answered this question yet.

See similar questions:

64
java.io.IOException: Failed to find null \ bin \ winutils.exe executable in Hadoop binaries. Eclipse spark on windows 7
thirty
Running Apache Hadoop 2.1.0 on Windows

or similar:

2453
How to install pip on windows?
2195
Is there an equivalent of "which" on the windows command line?
1975
How do I know which process the port is listening on in Windows?
1898
How do you claim that a specific exception is thrown in JUnit 4 tests?
12
Spark 1.6 - Could not find winutils binary in hadoop binary path
eleven
UnsatisfiedLinkError (NativeIO $ Windows.access0) when submitting mapreduce job to hadoop 2.2 from windows in ubuntu
one
Apache Nutch error: Injector: java.io.IOException: (null) command line entry: null chmod 0644
one
Could not find winutils.exe for hadoop 2.6.0 for 32-bit windows
0
Shell ERROR: 397 - Could not find winutils binary in hadoop binary path
0
Apache Apex minimum development environment

All Articles