HDFS says the file is still open, but the write process to it was killed

Question

I am new to hadoop and I spent the last couple of hours trying to deal with this problem, but I could not find anything that would help. My problem is that HDFS says the file is still open, although the writing process to it is long dead. This makes it impossible to read from the file.

I ran fsck in the directory, and it reports that everything is great. However, when I run "hadoop fsck -fs hdfs: // hadoop / logs / raw / directory_containing_file -openforwrite", I get

Status: CORRUPT
 Total size:    222506775716 B
 Total dirs:    0
 Total files:   630
 Total blocks (validated):  3642 (avg. block size 61094666 B)
  ********************************
  CORRUPT FILES:    1
  MISSING BLOCKS:   1
  MISSING SIZE:     30366208 B
  ********************************
 Minimally replicated blocks:   3641 (99.97254 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:   0 (0.0 %)
 Mis-replicated blocks:     0 (0.0 %)
 Default replication factor:    2
 Average block replication: 2.9991763
 Corrupt blocks:        0
 Missing replicas:      0 (0.0 %)
 Number of data-nodes:      23
 Number of racks:       1

By running fsck again in the file that opens, I get

.Status: HEALTHY
 Total size:    793208051 B
 Total dirs:    0
 Total files:   1
 Total blocks (validated):  12 (avg. block size 66100670 B)
 Minimally replicated blocks:   12 (100.0 %)
 Over-replicated blocks:    0 (0.0 %)
 Under-replicated blocks:   0 (0.0 %)
 Mis-replicated blocks:     0 (0.0 %)
 Default replication factor:    2
 Average block replication: 3.0
 Corrupt blocks:        0
 Missing replicas:      0 (0.0 %)
 Number of data-nodes:      23
 Number of racks:       1

Does anyone have any idea what is going on and how can I fix this?

Answer 1

, , -, , namenode , . , /. hdf https://twiki.grid.iu.edu/bin/view/Storage/HadoopRecovery (mirror: http://www.webcitation.org/5xMTitU0r)

: , - Scribe (, , DFSClient, Scribe), HDFS. HADOOP-6099 HDFS-278, . , , .