In ansible, how to synchronize 2 folders on the same remote machine?

Question

I have the following simple task:

copying everything in folder A to folder B. Since I have many hosts in a group, I use the following yaml task definition:

- name: Sync /etc/spark/conf to $SPARK_HOME/conf
  synchronize: src=/etc/spark/conf dest={{spark_home}}/conf
  delegate_to: "{{item}}"
  with_items: "{{play_hosts}}"
  tags: spark

However, running ansible-playbook gave me the following error:

TASK [cloudera : Sync /etc/spark/conf to $SPARK_HOME/conf] *********************
failed: [52.53.220.119 -> 52.53.200.0] (item=52.53.200.0) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.220.119:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.200.0", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}
failed: [52.53.200.193 -> 52.53.200.0] (item=52.53.200.0) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.200.193:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.200.0", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}
ok: [52.53.200.0 -> 52.53.200.0] => (item=52.53.200.0)
ok: [52.53.220.119 -> 52.53.220.119] => (item=52.53.220.119)
failed: [52.53.200.193 -> 52.53.220.119] (item=52.53.220.119) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.200.193:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.220.119", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}
failed: [52.53.200.0 -> 52.53.220.119] (item=52.53.220.119) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.200.0:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.220.119", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}
ok: [52.53.200.193 -> 52.53.200.193] => (item=52.53.200.193)
failed: [52.53.220.119 -> 52.53.200.193] (item=52.53.200.193) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.220.119:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.200.193", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: error in rsync protocol data stream (code 12) at io.c(226) [sender=3.1.0]\n", "rc": 12}
failed: [52.53.200.0 -> 52.53.200.193] (item=52.53.200.193) => {"cmd": "/usr/bin/rsync --delay-updates -F --compress --archive --rsh 'ssh -i /home/peng/.ssh/saphana.pem -S none -o StrictHostKeyChecking=no' --rsync-path=\"sudo rsync\" --out-format='<<CHANGED>>%i %n%L' \"/etc/spark/conf\" \"52.53.200.0:/opt/spark/spark-1.6.2-bin-hadoop2.4/conf\"", "failed": true, "item": "52.53.200.193", "msg": "Warning: Identity file /home/peng/.ssh/saphana.pem not accessible: No such file or directory.\nPermission denied (publickey).\r\nrsync: connection unexpectedly closed (0 bytes received so far) [sender]\nrsync error: unexplained error (code 255) at io.c(226) [sender=3.1.0]\n", "rc": 255}

Apparently, it looks like ansible is trying to create permutative pairs between all of my 3 hosts and synchronize between every pairs (so 9 rsync is performed), how do I avoid this and command ansible to only conduct rsync locally?

UPDATE: I've changed my task definition to use delegate.host:

- name: Sync /etc/spark/conf to $SPARK_HOME/conf
  synchronize: src=/etc/spark/conf dest={{spark_home}}/conf
  delegate_to: delegate.host
  tags: spark

But it is clearly not interpreted correctly by the ansible engine, the debugging log reveals that it is not substituted by the host IP address:

ESTABLISH SSH CONNECTION FOR USER: None

SSH: EXEC ssh -C -q -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o ConnectTimeout=10 -o ControlPath=/home/peng/.ansible/cp/ansible-ssh-%h-%p-%r delegate.host '/bin/sh -c '"'"'( umask 77 && mkdir -p "echo $HOME/.ansible/tmp/ansible-tmp-1470667606.38-157157938048153" && echo ansible-tmp-1470667606.38-157157938048153="echo $HOME/.ansible/tmp/ansible-tmp-1470667606.38-157157938048153" ) && sleep 0'"'"''

This looks like a deprecated feature, I'm using ansible 2.1.0.0

It looks like you need to replace delegate.host with real hostname or IP address. I have no idea at this moment, why it worked in my case, but it clearly is connected to scp_if_ssh = True setting in ansible.cfg without it I get an error too. — techraf

tribbloid tribbloid · Accepted Answer · 2016-08-08T16:48:31

Solved:

- name: Sync /etc/spark/conf to $SPARK_HOME/conf
  synchronize: src=/etc/spark/conf dest={{spark_home}} copy_links=true
  delegate_to: "{{ inventory_hostname }}"
  tags: spark

the delegate.host is probably removed in favour of the new variable

In ansible, how to synchronize 2 folders on the same remote machine?

1 Answers