How to scp to ec2 instance via ssm agent using boto3 and send file

Question

Hi need to transfer a file to ec2 machine via ssm agent. I have successfully installed ssm-agent in ec2 instances and from UI i am able to start session via "session-manager" and login to the shell of that ec2 machine.

Now I tried to automate it via boto3 and using the below code,

ssm_client = boto3.client('ssm', 'us-west-2') 
resp = client.send_command(
DocumentName="AWS-RunShellScript", # One of AWS' preconfigured documents
Parameters={'commands': ['echo "hello world" >> /tmp/test.txt']},
InstanceIds=['i-xxxxx'],
)

The above works fine and i am able to send create a file called test.txt in remote machine but his is via echo command Instead I need to send a file from my local machine to this remove ec2 machine via ssm agent, hence I did the following ,

Modified the "/etc/ssh/ssh_config" with proxy as below,

# SSH over Session Manager
host i-* mi-*
    ProxyCommand sh -c "aws ssm start-session --target %h --document-name AWS-StartSSHSession --parameters 'portNumber=%p'"

Then In above code, I have tried to start a session with below code and that is also successfully .

response = ssm_client.start_session(Target='i-04843lr540028e96a')

Now I am not sure how to use this session response or use this aws ssm session and send a file

Environment description: Source: pod running in an EKS cluster dest: ec2 machine (which has ssm agent running) file to be transferred: Important private key which will be used by some process in ec2 machine and it will be different for different machine's

Solution tried:

I can push the file to s3 in source and execute ssm boto3 libaray can pull from s3 and store in the remote ec2 machine
But I don't want to do the above due to the reason I don't want to store the private key i s3. So wanted to directly send the file from memory to the remote ec2 machine

Basically i wanted to achieve scp which is mentioned in this aws document : https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-working-with-sessions-start.html#sessions-start-ssh

It might be easier to "pull" the file into the instance. For example, if the file is stored in Amazon S3, then put a aws s3 cp command in the shell script. — John Rotenstein

Nathan Williams Nathan Williams · Accepted Answer · 2020-11-26T10:47:24

If you have SSH over SSM setup, you can just use normal scp, like so:

scp file.txt ec2-user@i-04843lr540028e96a

If it isn't working, make sure you have:

Session Manager plugin installed locally
Your key pair on the instance and locally (you will need to define it in your ssh config, or via the -i switch)
SSM agent on the instance (installed by default on Amazon Linux 2)
An instance role attached to the instance that allows Session Manager (it needs to be there at boot, so if you just attached, reboot)

Reference: https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager-getting-started-enable-ssh-connections.html

If you need more detail, give me more info on your setup, and I'll try and help.

How to scp to ec2 instance via ssm agent using boto3 and send file

1 Answers