Geek Answers Handbook

The hdfs dfs command is slow - is there a way to make this faster?

I am on Hadoop 2.2.0, starting the installation of Single Node.

I understand that it is hdfs dfs -lsslow because it expands JVMevery time it is called.

Is there a way to get the JVM to work so that simple commands can complete faster?

+4
source share
4 answers

In command-line commands, pigs, such as fs -ls, work quite quickly, so this can be a pragmatic workaround. The problem is that when working with other teams this will not work.

, script , : https://unix.stackexchange.com/a/144722/46085. , , script ( grunt ), - . head , , , , .

, , . , .

+1

Hadoop fs -ls , 20 . , Wi-Fi, .

, hadoop, Wi-Fi. , , .

0

Hadoop Tools. hdfs dfs, . HDFS bash, .

put.

0

, .

- HDFS Shell HDFS .

https://github.com/avast/hdfs-shell

  • HDFS DFS JVM , HDFS Shell , , HDFS.
  • Commands can be used in a short way - for example. hdfs dfs -ls /, ls /- both will work
  • Ending an HDFS Path Using the Tab Key
  • we can easily add any other HDFS manipulation function
  • there is a history of commands stored in the history log (~ / .hdfs-shell / hdfs-shell.log)
  • support for relative directory + commands cdandpwd
0
source

More articles:


All Articles