Hadoop on windows: file not found exception

Question

I use hasoop in windows and I configured everything well (installing cygwin, no ssh password, etc.)

I compiled the wordcount program in WC.jar and tried to execute. It works great offline. But in fully distributed mode, it gives a FileNotFoundException

Please look in the magazines and tell me what is wrong with him. I started using dfs and mapreduce in MACH1. (this is my master)

 $ bin/hadoop jar WC.jar WordCount words result 10/07/24 16:57:38 INFO input.FileInputFormat: Total input paths to process : 2 10/07/24 16:57:39 INFO mapred.JobClient: Running job: job_201007241657_0001 10/07/24 16:57:40 INFO mapred.JobClient: map 0% reduce 0% 10/07/24 16:57:50 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00003_0, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_0/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:57:55 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 00002_0, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000002_0/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:07 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00003_1, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_1/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:14 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00003_2, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000003_2/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:26 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00002_0, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_0/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:34 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_r_0 00001_0, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-SYSTEM/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_r_000001_0/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:41 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00002_1, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_1/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:47 INFO mapred.JobClient: Task Id : attempt_201007241657_0001_m_0 00002_2, Status : FAILED java.io.FileNotFoundException: File C:/tmp/hadoop-328510/mapred/local/taskTracke r/jobcache/job_201007241657_0001/attempt_201007241657_0001_m_000002_2/work/tmp d oes not exist. at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSys tem.java:361) at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem. java:245) at org.apache.hadoop.mapred.TaskRunner.setupWorkDir(TaskRunner.java:519) at org.apache.hadoop.mapred.Child.main(Child.java:155) 10/07/24 16:58:53 INFO mapred.JobClient: Job complete: job_201007241657_0001 10/07/24 16:58:53 INFO mapred.JobClient: Counters: 0 328510@01HW179531 /usr/local/hadoop-0.20.2 $` 

Thanks.

Answer 1

I think I saw this exception before, but I don't have access to my old magazines to confirm this. I decided to remove FileNotFoundException using reformatted namenode . You might want to check the namenode logs for an "inconsistent state" to confirm the reason before reformatting.