Copy a File From S3 to EC2 Automatically

Question

It's being a while I'm struggling with this situation:

I have a local application in PHP (WordPress)
When I commit my changes I send the changed files automatically to S3 via Git Hooks
S3 Bucket receives these changed files
Right after this, I send a notification to a lambda function when the file is included/deleted
In the lambda function, using javascript for instance, I have identified the bucket and file names (So far so good).

The question is: How can I copy this file from the Bucket to the EC2 to complete my automation ?

What I have tried so far:

A) Manually speaking I can do something like this inside my ec2 instance, that works perfectly:

aws s3 sync s3://my-bucket "\var\www\html*" --exclude "wp-config.php"

B) Also I have executed one similar command on "SSM Run Command" like this that also works perfectly:

wget https://my-bucket.s3.amazonaws.com/file.txt

C) Trying to create a very simple SSM document that execute a simple random bash script like I have done in the previous attempt, trying to create the document - create command or session - session type and trying some versions of what you can see in this example (with/whitout quotations...):

---
schemaVersion: "2.2"
description: "Command Document Example JSON Template"
parameters:
  Message:
    type: "String"
    description: "Example"
    default: "Hello World"
mainSteps:
- action: "aws:runPowerShellScript"
  name: "example"
  inputs:
    runCommand:
      - cd /var/www/html
      - wget https://my-bucket.s3.amazonaws.com/file.txt

With this error: failed to run commands: fork/exec /usr/bin/pwsh: no such file or directory (even trying with and without quotes)

If someone have some idea, maybe a code somewhere to help me automate the copy of a single file from S3 to EC2 this would be very apreciated !!!

Thank You guys in advance ! :)

Run Command seems as a good fit. Do you use linux? Why not use run command for linux? Do you need powershell? — Marcin
The error you get indicates that you don't have powershell installed. Should try AWS-RunShellScript document. — Marcin
Marcin, thank you so much, you pointed me to the right direction ! I'll be trying tomorrow execute this on Lambda (with the proper permissions etc...) Tks :) — Carlos Valença de Paula
At the very end, after lots of tries I was able to solve this issue with this link: stackoverflow.com/questions/60470604/… the comand in this post the way it is was the solution ! But thank you very much for all your help !!! — Carlos Valença de Paula

Mauricio Barrera Mauricio Barrera · Accepted Answer · 2020-05-19T00:02:25

I think you almost got it, you can try execute the SSM document using lambda and document should execute the AWS CLI inside the instance.

Things to consider:

The instance should have the correct permissions (IAM Role) to download the file from S3
The instance should have AWS CLI installed on it
The instance should be part of SSM Inventory (you will need also an IAM Role)

Also I'm not sure why are you trying to copy a file in a block storage that is already in S3, I think is not cost efficient.

Copy a File From S3 to EC2 Automatically

1 Answers