Using ec2_snapshot getting error message “boto required for this module”

Question

I am using Ubuntu Server 16.4 with ansible 2.4 on AWS

My playbook is trying to take snapshots of ec2 vol. See below

- hosts: localhost
  connection: local
  become: yes
  become_method: sudo
  gather_facts: yes
  any_errors_fatal: True

- name: Take snapshots of all volume"
  ec2_snapshot:
      volume_id: "{{item.id}}"
      description: "Taken on {{ ansible_date_time.date }}"
      snapshot_tags:
          frequency: hourly
  with_items: "{{ aws_ec2_vol_setting }}"

I run the playbook with the following cmd

ansible-playbook -vvv  pb_aws_backup_nw_us_sat.yml

Using module file /usr/lib/python2.7/dist-packages/ansible/modules/cloud/amazon/ec2_snapshot.py
<127.0.0.1> ESTABLISH LOCAL CONNECTION FOR USER: r_ansible
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
<127.0.0.1> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036 `" && echo ansible-tmp-1517934063.54-197986659054036="` echo /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036 `" ) && sleep 0'
<127.0.0.1> PUT /tmp/tmprERpt_ TO /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036/ec2_snapshot.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036/ /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036/ec2_snapshot.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'sudo -H -S -n -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-ociuiywkpfvurbbesjwxhczxoglttlsa; /usr/bin/python /home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036/ec2_snapshot.py; rm -rf "/home/r_ansible/.ansible/tmp/ansible-tmp-1517934063.54-197986659054036/" > /dev/null 2>&1'"'"' && sleep 0'
Using module file /usr/lib/python2.7/dist-packages/ansible/modules/cloud/amazon/ec2_snapshot.py
<127.0.0.1> EXEC /bin/sh -c 'echo ~ && sleep 0'
The full traceback is:
  File "/tmp/ansible_gUIlz4/ansible_module_ec2_snapshot.py", line 127, in <module>
    import boto.ec2

failed: [localhost] (item={u'vol': u'vol-us-sat-01', u'id': u'vol-0b6aaa3b8289580f6', u'server': u'us-nv-sat-01'}) => {
    "changed": false,
    "invocation": {
        "module_args": {
            "aws_access_key": null,
            "aws_secret_key": null,
            "description": null,
            "device_name": null,
            "ec2_url": null,
            "instance_id": null,
            "last_snapshot_min_age": 0,
            "profile": null,
            "region": null,
            "security_token": null,
            "snapshot_id": null,
            "snapshot_tags": {
                "frequency": "hourly"
            },
            "state": "present",
            "validate_certs": true,
            "volume_id": "vol-0b6aaXXXXXXXX",
            "wait": true,
            "wait_timeout": 0
        }
    },
    "item": {
        "id": "vol-0b6aXXXXXXXXXX",
        "server": "us-nv-sat-01",
        "vol": "vol-us-sat-01"
    },
    "msg": "boto required for this module"
}

Note: The access key and secret key are empty because i am using a IAM role that is assign to the server.

I have check my host and to me at least it looks like i have all the requirements.

$ which python
/usr/bin/python
$ pip list boto | grep boto
boto (2.48.0)
boto3 (1.5.23)
botocore (1.8.37)
$ python -V
Python 2.7.12

python modules all seem to be there and importing fine is well

$ python
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import boto
>>> import boto.ec2
>>> import boto3

So i am not sure why i am getting the error "boto required for this module". I have also tried without any success the solution suggested here but I still get the issue.

Answer 1

It might be the case that the pip packages you are seeing somehow are not able to run properly with the default python that is installed (if you can run sudo pip freeze | grep boto you can see the full list of packages), if boto is not there maybe you'd need to install it using sudo pip install. On the other hand something I'd rather do is to set up a python virtual env and install all packages Ansible requires only in that isolated env, like:

Dependencies: sudo apt-get install python-setuptools sudo apt-get install python-pip sudo pip install virtualenv

Then create the virtual env: virtualenv ansible_vEnv

Activate the virtual env: source ansible_vEnv/bin/activate

Then just install all python dependencies with pip for the ec2.py:

• boto > 2.45
• boto3 > 1.5 (not sure if this one's required though)
• botocore

Hope it helps!

Answer 2

I would suggest you uninstall ansible with apt-get and install it with pip.

sudo apt uninstall ansible
sudo apt install gcc python-dev python-pip
sudo pip install --upgrade PyCrypto ansible awscli boto boto3 ansible-role-manager ansible-playbook-debugger retry

Answer 3

What @Konstantin Suvorov suggest was correct. Boto was not install for root/sudo

Not sure how I did it but pip and boto were install under the local user at: ~/.local/lib/python2.7/site-packages/

When i login as root and try to run pip to install boto i go the following error:

    root$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2927, in <module>
    @_call_aside
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2913, in _call_aside
    f(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 637, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 650, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 829, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==9.0.1' distribution was not found and is required by the application

to find what version of pip root was using I did the following: $python -c "import pip; print(pip.__version__)

and found it was pip version 8.1.1. I then updated the following file: $vi /usr/local/bin/pip

 #!/usr/bin/python
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==9.0.1','console_scripts','pip'
__requires__ = 'pip==9.0.1'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('pip==9.0.1', 'console_scripts', 'pip')()
    )

and change all reference of 9.0.1 to 8.1.1. This allowed pip to work while logged in as root. I then updated pip using $"sudo -H pip install --upgrade pip and the checking with $python -c "import pip; print(pip.__version__) shows the correct version for root and ansible user

This meant all pip installed on the box are now the same version. I am sure this is not a good solution and would be better for all users (ansible and root) to point to the same pip install but i am not sure how to do this.

with pip working i was able to install boto and boto3 pip install boto and pip install boto3