Hdfs dfs -put with rewrite?

Question

I use

hdfs dfs -put myfile mypath 

and for some files I get

 put: 'myfile': File Exists 

• Does this mean that there is a file with the same name, or does it mean that the same exact file (size, content) already exists?
• How can I specify the -overwrite option here?

Thanks!

Answer 1

put: 'myfile': File Exists

So a file named "myfile" already exists in hdf. You cannot have multiple files with the same name in hdfs

You can overwrite it with hadoop fs -put -f /path_to_local /path_to_hdfs

Answer 2

You can overwrite a file in hdfs with the -f command. for instance

 hadoop fs -put -f <localfile> <hdfsDir> 

OR

 hadoop fs -copyFromLocal -f <localfile> <hdfsDir> 

It worked for me. However, the -f command will not work with get or copyToLocal. check this question

Answer 3

• A file with the same name exists in the place you are trying to write.
• You can overwrite by specifying the -f flag.