how to copy file from remote server to HDFS

Question

I have a remote server and servers authenticated Hadoop environment.

I want to copy file from Remote server to Hadoop machine to HDFS

Please advise efficient approach/HDFS command to copy files from remote server to HDFS.

Any example will be helpful.

as ordinary way to copy file from remote server to server itself is

scp -rp file remote_server:/tmp

but this approach not support copy directly to hdfs

Answer 1

Here the remote server you mean to say it is not in the same network as the hadoop nodes. If that is the case may be you can scp from remote machine to hadoop nodes local file system and then use -put or -copyFromLocal command to move to HDFS.

example: hadoop fs -put file-name hdfs://namenode-uri/path-to-hdfs

Answer 2

You can try that:

ssh remote-server "hadoop -put - /tmp/file" < file