I am trying to run a simple JUnit test with Hadoop 2.2. on my Windows 7 machine. I have an example test available at http://grepalex.com/2012/10/20/hadoop-unit-testing-with-minimrcluster/ , so I use ClusterMapReduceTestCase as the base class to run MiniDFSCluster .
When I run the test, I get (in the console):
2013-11-15 11: 59: 24,636 ERROR [main] util.Shell (Shell.java:getWinUtilsPath (303)) - Could not find winutils binary in hadoop binary path java.io.IOException: Could not find null executable \ bin \ winutils.exe in the Hadoop binaries. at org.apache.hadoop.util.Shell.getQualifiedBinPath (Shell.java:278) at org.apache.hadoop.util.Shell.getWinUtilsPath (Shell.javahaps00) on org.apache.hadoop.util.Shell. (Shell.java:293) at org.apache.hadoop.util.StringUtils. (StringUtils.java:76) (...)
and at the output of JUnit:
java.lang.UnsatisfiedLinkError: org.apache.hadoop.io.nativeio.NativeIO $ Windows.access0 (Ljava / lang / String; I) Z on org.apache.hadoop.io.nativeio.NativeIO $ Windows.access0 (native method ) on org.apache.hadoop.io.nativeio.NativeIO $ Windows.access (NativeIO.java:435) on org.apache.hadoop.fs.FileUtil.canWrite (FileUtil.java:996) at org.apache.hadoop. hdfs.server.common.Storage $ StorageDirectory.analyzeStorage (Storage.java:451) at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverStorageDirs (FSImage.java:282) at org.apache.hadoop.hdfs. server.namenode.FSImage.recoverTransitionRead (FSImage.java:200) (...)
I read the topics: I could not find the winutils binary in the hadoop binary path and Starting Apache Hadoop 2.1.0 on Windows I also examined the possible solutions available at http://www.srccodes.com/p/article/38/build-install -configure-run-apache-hadoop-2.2.0-microsoft-windows-os and http://www.srccodes.com/p/article/39/error-util-shell-failed-locate-winutils-binary-hadoop -binary-path
Unfortunately, this does not work with JUnit tests. If I write junit tests, I want them to be independent of the development environment (so that this can work without problems on every machine for developers). Therefore, I use maven, and I do not want to create the necessary libraries myself and put them in the Hadoop bin folder, etc.
Any suggestions what would be the best solution in this case?
java windows junit hadoop
Jakub
source share