Geek Answers Handbook

Hadoop copyFromLocal problem with copying directory

I would like to copy the entire local directory with some subdirectories and files to HDFS. HDFS already contains a root directory and some subdirectories with files. I just want to add newer files from the local directory.

Local directory /www/hitlogfetcher/logs/: day=20--hour=00/files..... |-hour=01/files..... |-hour=02/files.... HDFS /hitlogfetcher-test/: day==20--hour=00/files |-hour=01/

When I used the command: hadoop dfs -copyFromLocal / www / hitlogfetcher / logs / * / hitlogfetcher-test / I received an error message:

 Target /hitlogfetcher-test/day=20 is a directory
  • day = 20 is a directory containing some subdirectories and files

So, I would like to copy the files from the hour = 01 directory, and then to the cpy hour = 02 directory and its files.

Is this possible using debugger shell commands or in another way?

Thanks Michal

0
source share
1 answer

The copyFromLocal command will load directories recursively by default, so you do not need "*":

 hadoop dfs -copyFromLocal /www/hitlogfetcher/logs/ /hitlogfetcher-test/ ^
0
source

More articles:


All Articles