rsync code will run, but not in cron

Question

I have a web server (odin) and a backup server (jofur). On jofur, I can run the following code to rsync my web directories (via key authentication) from odin to jofur:

rsync -avz -e ssh [email protected]:/home/backups /home/myuser/odin

If I enter this into the command line, everything rsyncs perfectly:

myuser@jofur:~$ rsync -avz -e ssh [email protected]:/home/backups /home/myuser/odin

receiving incremental file list

sent 23 bytes  received 1921 bytes  1296.00 bytes/sec
total size is 349557271  speedup is 179813.41

I want this to run every morning, so I edited my crontab to read this:

0 4 * * * rsync -avz -e ssh [email protected]:/home/backups /home/myuser/odin

This doesn't work. The following message is deposited in /var/mail/myuser:

Could not create directory '/home/myuser/.ssh'. Host key verification failed. rsync: connection unexpectedly closed (0 bytes received so far) [Receiver] rsync error: unexplained error (code 255) at io.c(605) [Receiver=3.0.9]

I'm not sure what this error means. I'm wary of futzing blindly with permissions because I don't want to leave any backdoors open. Any suggestions?

Answer 1

Its hard to tell whether cron is using the wrong rsync binary or whether rsync requires some variable which is not being set in cron. Please set the stdout/stderr as shown below and pass on the output of the log file

Also, try doing a "which rsync" from the command line ; this will tell you which rsync you are using from the command line.

0 4 * * * rsync -avz -e ssh [email protected]:/home/backups /home/myuser/odin > /tmp/cron_output.log 2>&1

EDIT :

Can you create a shell script called SOME_DIR/cron_job_rsync.sh which contains the following. Make sure you set the execute bit.

#!/bin/sh
/usr/sbin/rsync -avz -e ssh [email protected]:/home/backups /home/myuser/odin

And modify the cronjob as shown below

0 4 * * * SOME_DIR/cron_job_rsync.sh >/tmp/cron_output.log 2>&1

Answer 2

I had a similar issue. Mine was the HOME directory was encrypted.

If your user is logged, it works the known_hosts.

But when it's a cron, the cron uses the right user BUT it does not have access to your $HOME/~/.ssh directory because is encrypted :-(

Answer 3

i got the same error just like you.

I finally found user home directory is an 'mount point', when logged in, it changed.

You can use the shell command 'mount' to check if you have the same way to use home directory.

So, i logged in and 'cd /', then do

```

cp -ar ${HOME}/.ssh /tmp/
sudo umount ${HOME}
mv /tmp/.ssh ${HOME}

```

There is may failed, because you need to check the ${HOME} if you have the right to write, if not, try sudo or add writable to ${HOME}.

After that, every thing being fine.

Answer 4

Please follow the below steps to avoid the error http://umasarath52.blogspot.in/2013/09/solved-rsync-not-executing-via-cron.html

Answer 5

I resolved this issue by communicating with the administrators for my server. Here is what they told me:

For advanced security and performance, we use 1H (Hive) which utilizes a chrooted environment for users. Libraries and binaries should be copied to the chrooted environment to make them accessible.

They sent me a follow up email telling me that the "Relevent" packages have been installed. At that point, the problem was resolved. Unfortunately, I didn't get any additional information from them. The host was Arvixe, but I'm guessing that anyone using 1H (Hive) will encounter a similar problem. Hopefully this answer will be helpful.

Answer 6

Use the rrsync script together with a dedicated ssh key as follows:

REMOTE server

mkdir ~/bin
gunzip /usr/share/doc/rsync/scripts/rrsync.gz -c > ~/bin/rrsync
chmod +x ~/bin/rrsync

LOCAL computer

ssh-keygen -f ~/.ssh/id_remote_backup -C "Automated remote backup"      #NO passphrase
scp ~/.ssh/id_remote_backup.pub [email protected]:/home/devel/.ssh

REMOTE computer

cat id_remote_backup.pub >> authorized_keys

Prepend to the newly added line the following

command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding

So that the result looks like

command="$HOME/bin/rrsync -ro ~/backups/",no-agent-forwarding,no-port-forwarding,no-pty,no-user-rc,no-X11-forwarding ssh-rsa AAA...vp Automated remote backup

LOCAL

Put in your crontab the following script with x permission:

#!/bin/sh
echo ""
echo ""
echo "CRON:" `date`
set -xv
rsync -e "ssh -i $HOME/.ssh/id_remote_backup" -avzP [email protected]:/ /home/user/servidor 

Source: http://www.guyrutenberg.com/2014/01/14/restricting-ssh-access-to-rsync/

Answer 7

I did several steps to make it work.

1. Check your paths. For every command you'll use check which [command] and use that full path for the crontab

2. Open crontab as the user you want to run it with so it has access to that users ssh-key

3. Add (remember to user which) ssh-agent && [your ssh-command] so it can connect over ssh.

4. When authentication still fails at this point. Try to generate a passwordless ssh-key. This way you can skip the password prompting.

For debugging it is useful to add -vvv to the ssh command in rsync. It makes it clear what goes wrong.

Answer 8

Using the correct keyring solved the issue for me. Add the following line to your crontab:

SSH_AUTH_SOCK=/run/user/1000/keyring/ssh

In total, your crontab (edited by calling crontab -e from your terminal) should now look as follows:

SSH_AUTH_SOCK=/run/user/1000/keyring/ssh
0 4 * * * rsync -avz [email protected]:/home/backups /home/myuser/odin

Background: It turns out that some Linux distributions use a keyring to protect your public-private key pair - so the key pair is password-protected without ever noticing you. Consequently, rsync is not able to open your ssh key for authentication.

Note that I also omitted the -e ssh; I think it is not necessary here.

Further Troubleshooting: rsync does not provide a lot of debugging output. What helped me identify the problem was to put a dummy scp command, which is much more verbose, in my crontab. A crontab entry for troubleshooting may look something like:

* * * * * scp -v [email protected]:/home/backups/dummy.txt /home/myuser/odin/dummy.txt >> /home/myuser/odin/dummy.txt.log 2>&1

The above command will run every minute (great for developing) and it will copy a file /home/backups/dummy.txt to your local machine. All logs (stdout and stderr) are written to to /home/myuser/odin/dummy.txt.log. Inspect these logs to see where the error precisely comes from.

Reference: The troubleshooting explained above lead me to the solution: https://unix.stackexchange.com/a/332353/395749