How can I list subdirectories recursively for HDFS?

Question

I have a set of directories created in HDFS recursively. How can I list all directories? For a normal unix file system, I can do this using the following command

find /path/ -type d -print 

But I want to get similar for HDFS.

Answer 1

You can use the hadoop dfs -lsr /dirname command recursively to display the contents of a directory.

To filter directories only, you can grep "drwx" (since the owner has rwx permission for directories) in the output of the command above.

Therefore, the whole team will look like the one shown below.

 $hadoop dfs -lsr /sqoopO7 | grep drwx 

Answer 2

The answer given by @Shubhangi Pardeshi is correct, but the version is outdated for the latest suoop command. So the new last command can be used as below

 hdfs dfs -ls -R /user | grep drwx