I am using GNU Parallel to download 1000 files from S3 to two EC2 hosts (-S:,R), where 'R' is IP of remote peer.
With --jobs 64, it takes a long time to download 64 files (rather large files) from S3:
time parallel --jobs 64 --joblog out.txt -S:,R --env HTTPS_PROXY --env S3_DIR "aws s3 cp ${S3_DIR}part-{}.gz /my/path/" ::: `seq -f "%05g" 0 1000`
As the result, I get
Connection to R closed by remote host.
This is due to sshd configuration on the remote peer, and I cannot change these settings (machines are not mine.)
Looking at the documentation, I cannot find anything like keep-alive. How can I work around this issue? Thank you.