Hadoop fs block size search?

Question

In Hadoop fs, how do I look for the block size for a specific file?

First of all, I was interested in the command line, for example:

hadoop fs ... hdfs://fs1.data/... 

But it does not seem to exist. Is there a Java solution?

Answer 1

Hasoop fs doesn't seem to have parameters for this.

But hadoop fsck could.

You can try this

 $HADOOP_HOME/bin/hadoop fsck /path/to/file -files -blocks 

Answer 2

The fsck commands in the other answers list the blocks and let you see the number of blocks. However, to see the actual block size in bytes at no extra cost, do:

 hadoop fs -stat %o /filename 

Default block size:

 hdfs getconf -confKey dfs.blocksize 

Unit Details

Unit sizes are not documented in the hadoop fs -stat , however, looking at the source line and documents using the hadoop fs -stat method, we see that it uses bytes and cannot report block sizes greater than 9 exabytes.

The units for the hdfs getconf cannot be bytes. It returns any string used for dfs.blocksize in the configuration file. (This is seen in the source for the final function and its indirect caller )

Answer 3

I think this should be doable with

 hadoop fsck /filename -blocks 

but I get Connection refused