executable file not found in $PATH when trying to connect to a container in an AWS ECS EC2 instance

Question

I am having a hard to figuring out how to connect to my Docker container inside my EC2 instance via ssh. Basically, I can get into the server okay and list my container using docker ps.

However, I can't get a shell inside of that container to run some shell scripts I need to get it running.

Locally, I just use ...

docker attach [cid] # or 'docker exec -it [cid] bash' to open new shell

... but there are obvious issues with running docker attach and with docker exec -it [cid] bash I am getting ...

rpc error: code = 2 desc = "oci runtime error: exec failed: exec: \"bash\": executable file not found in $PATH"

To provide some detail I believe I have set up ECS correctly. From the applicable community AMI named amzn-ami-2016.03.e-amazon-ecs-optimized I have completed the following set up:

An applicable ecsInstanceRole and ecsServiceRole
A running EC2 instance.
A Load Balancer pointing to that instance.
A tagged repository in ECS.
A ECS Task Definition pointing to ECS repository.
A ECS Cluster with one instance slot allocated and has been successfully associated to the EC2 via(I believe) the start-up script. This cluster also has a service associated with the load balancer I mentioned as well as the associated task definition.

I don't believe this is where my issue lies(but maybe)

Perhaps it's my Dockerfile?

FROM    centos:centos6

RUN yum -y update; yum clean all; \
    yum groupinstall -y "Web Server" "MySQL Database" "PHP Support" "Development Tools"; \
    service httpd start; \
    chkconfig httpd on;

RUN yum install -y openssh openssh-clients git php-mysql php-gd php-imap php-ldap php-odbc php-pear php-xml php-xmlrpc php-mbstring php-mcrypt php-mssql php-snmp php-soap php-tidy curl curl-devel php-pecl-apc mysql;

# PHP
RUN sed -i '\%^<Directory "/var/www/html">%,\%^</Directory>% s/AllowOverride None/AllowOverride All/' /etc/httpd/conf/httpd.conf;

# MONGO
ADD docker/mongo/setup.sh /root/mongo.setup.sh

RUN chmod +x /root/mongo.setup.sh;

RUN yum install -y cyrus-sasl2 cyrus-sasl-devel php-devel; \
    echo "extension=mongo.so" >> /etc/php.ini;

# GIT CONFIG AND AUTH
ADD docker/ssh/ /root/.ssh/

RUN chmod 600 /root/.ssh/*; \
    touch /root/.ssh/known_hosts; \
    ssh-keyscan github.com >> /root/.ssh/known_hosts;

# EMAIL SES CONFIG AND AUTH
ADD docker/postfix/sasl_passwd /etc/postfix/sasl_passwd

ADD docker/postfix/main.appended.txt /etc/postfix/main.appended.txt

ADD docker/postfix/setup.sh /root/postfix.setup.sh

RUN chmod +x /root/postfix.setup.sh;

RUN yum install -y stunnel telnet telnet-server mailx postfix cyrus-sasl cyrus-sasl-plain cyrus-sasl-md5 cyrus-imapd postfix cyrus-sasl cyrus-sasl-plain cyrus-sasl-md5 cyrus-imapd openssl openssl-devel; \
    sed -i 's/-o smtp_fallback_relay=/#-o smtp_fallback_relay=/g' /etc/postfix/master.cf; \
    cat /etc/postfix/main.appended.txt >> /etc/postfix/main.cf; \
    chmod 600 /etc/postfix/main.appended.txt; \
    postmap hash:/etc/postfix/sasl_passwd;

# ADD YII LIBRARY
ADD docker/yii.tar.gz /var/www

# CLONE APPLICATION
RUN cd /var/www; \
    mkdir repo; \
    git clone [email protected]:myrepos.git html; \
    cd html; \
    git checkout production;

# NODE INSTALLATION
RUN yum install -y gcc gcc-c++ wget tar; \
    cd /root/; \
    wget http://nodejs.org/dist/v0.10.30/node-v0.10.30.tar.gz; \
    tar xzvf node-v*; \
    cd node-v*; \
    ./configure; \
    make; \
    make install;

# CREATE SOME DIRECTORIES FOR THE APPLICATION
RUN mkdir /var/www/.tmp; \
    mkdir /var/www/.tmp/data; \
    mkdir /var/www/coach_tests; \
    mkdir /var/www/html/protected/data/sessions; \
    mkdir /var/www/html/staging/protected/data/sessions; \
    mkdir /var/www/html/development/protected/data/sessions; \
    mkdir /var/www/html/protected/runtime; \
    mkdir /var/www/html/staging/protected/runtime; \
    mkdir /var/www/html/development/protected/runtime; \
    mkdir /var/www/html/assets; \
    mkdir /var/www/html/images/cache;

# GRUNT SETUP
ADD docker/www/Gruntfile.coffee /var/www/Gruntfile.coffee
ADD docker/www/package.json /var/www/package.json

RUN npm install -g grunt-cli; \
    cd /var/www; \
    npm install;

# S3 MOUNT CONFIG AND AUTH
ADD docker/s3/setup.sh /root/s3.setup.sh

RUN chmod +x /root/s3.setup.sh;

RUN yum install -y glib2-devel fuse-devel libevent-devel libxml2-devel; \
    cd /root/; \
    wget https://github.com/downloads/libevent/libevent/libevent-2.0.21-stable.tar.gz; \
    tar -xzf libevent-2.0.21-stable.tar.gz; \
    cd libevent-2.0.21-stable; \
    ./configure && make; \
    make install; \
    echo "/usr/local/lib/" > /etc/ld.so.conf.d/riofs.conf; \
    export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig; \
    ln -s /usr/local/lib/libevent-2.0.so.5 /usr/lib64/libevent-2.0.so.5; \
    cd ../; \
    git clone https://github.com/skoobe/riofs.git; \
    cd riofs; \
    ./autogen.sh; \
    ./configure; \
    make; \
    make install; \
    mkdir -p /var/www/html/images/user; \
    mkdir -p /var/www/html/images/store; \
    mkdir -p /var/www/html/images/sponsor; \
    mkdir -p /var/www/html/images/supporter; \
    mkdir -p /var/www/html/images/media;

# USEFUL TOOLS
ADD docker/startup.sh /root/startup.sh

RUN chmod +x /root/startup.sh;

RUN yum install -y nano;

# GRUNT INITIATE
RUN cd /var/www; \
    grunt init;

How exactly are you trying to connect to the containers? And what is it you're using Docker for? That's a lot of stuff in there. — ldg
We should separate ECS specific issues from those coming from your specific container settings. First off, are you able to successfully run this command in your ECS instance - docker run -it centos:centos6 bash? (Worked for me on our ECS instance) — Yaron Idan

Mitchell Mitchell · Accepted Answer · 2016-07-17T15:17:48

Oh man... sometimes when I am new to something I spend so much time looking for the small details that I gleam right over the more obvious issue. In this particular case I ignored the "IMAGE" column in my docker ps listing assuming that it must have been my image without paying attention to anything else but the container ID. The only running container was the amazon/amazon-ecs-agent:latest so no wonder I had issues getting a shell inside it.

It took Yaron's comment to remind me of this, as the docker run command was mentioned triggering an ah-ha moment. I guess, I assumed it ran after the Task Definition thus the very reason I thought the only running container had to mine :P

After running docker run -it -p 88:80 --privileged --cap-add MKNOD --cap-add SYS_ADMIN -d 1234546677788.dkr.ecr.us-west-2.amazonaws.com/my-ecs-repos /bin/bash on the ECS EC2 instance, I was able visit the website and get a shell inside of the container :D

executable file not found in $PATH when trying to connect to a container in an AWS ECS EC2 instance

2 Answers