hdfs dfs -put with overwrite?

Question

I am using

hdfs dfs -put myfile mypath

and for some files I get

put: 'myfile': File Exists

• does that mean there is a file with the same name or does that mean the same exact file (size, content) is already there?
• how can I specify an -overwrite option here?

Thanks!

Answer 1

put: 'myfile': File Exists

Means,the file named "myfile" already exists in hdfs. You cannot have multiple files of the same name in hdfs

You can overwrite it using hadoop fs -put -f /path_to_local /path_to_hdfs

Answer 2

You can overwrite your file in hdfs using -f command.For example

hadoop fs -put -f <localfile> <hdfsDir>

OR

hadoop fs -copyFromLocal -f <localfile> <hdfsDir>

It worked fine for me. However -f command won't work in case of get or copyToLocal command. check this question

Answer 3

1. A file with the same name exists at the location you're trying to write to.
2. You can overwrite by specifying the -f flag.

Answer 4

Just updates to this answer, in Hadoop 3.X the command a bit different

hdfs dfs -put -f /local/to/path hdfs://localhost:9870/users/XXX/folder/folder2