Use cf push with Amazon ECR?

Question

I am attempting to execute the Cloud Foundry CLI command: cf push and have it pull a docker image from our aws ecr repository. I keep getting this error message:

Failed getting docker image by tag: unauthorized: access to the requested resource is not authorized

I have logged into aws via the aws configure command with an Access Key ID and a Secret Access Key, and I have gotten the ecr login from the 'aws ecr get-login' command.

I am able to pull the image using docker:

docker pull 1234567.dkr.ecr.us-east-1.amazonaws.com/repo/example:17.5.0-SNAPSHOT

So I know I have valid credentials. However, this is the error when attempting the cf push command:

CF_DOCKER_PASSWORD=mypassword cf push example --docker-image 1234567.dkr.ecr.us-east-1.amazonaws.com/repo/example:17.5.0-SNAPSHOT --docker-username myusername
Using docker repository password from environment variable CF_DOCKER_PASSWORD.
....<snip>
Staging process started ...
Failed getting docker image by tag: unauthorized: access to the requested resource is not authorized  Going to retry attempt: 1
Failed getting docker image by tag: unauthorized: access to the requested resource is not authorized  Going to retry attempt: 2
Failed getting docker image by tag: unauthorized: access to the requested resource is not authorized  Going to retry attempt: 3
Failed getting docker image by tag: unauthorized: access to the requested resource is not authorized
Staging process failed: Exit trace for group:
builder exited with error: failed to fetch metadata from [repo/example] with tag [17.5.0-SNAPSHOT] and insecure registries [] due to unauthorized: access to the requested resource is not authorized
Exit status 2
Staging Failed: Exited with status 2
Destroying container
Successfully destroyed container
...<snip>

I have tried my aws login and password, and I have also tried the Access Key ID and the Secret Access Key. Both return the same error.

Any ideas on how to integrate aws ecr with cf push?

Is there any change your password contains a symbol or special character that gets interpreted by your shell? Do you get the same error if you enter the password interactively? — dkoper
I have tried setting the environment variable for the password using the export command instead of it being inline. But it did not make a difference. I don't see how to make it request the password interactively. There are definitely some funky characters in the password. I also tried quotes around the password, but got the same result. — Will Turner
On cf CLI 6.28.0 or higher, if a Docker image and username are specified to cf push and environment variable CF_DOCKER_PASSWORD is not set, the user is prompted for the password. BTW, the private Docker repo feature requires a target CF release of v260 (CC API v2.82.0) onwards. — dkoper
The AWS user that you are using, did you create the ECR instance using that same user? If not, does the policy associated for that user grant you access to the ECR instance? — K.AJ
I did not create the ECR instance, but I have been given access to it. As shown above, I can do a docker pull on the image from the repository. I think part of the problem is the --docker-username and password. The ECR login requires going through the aws get-login and then the docker login process. I suspect that the cf docker image pull does not use the login generated by docker login and it can't use either my username or my access key. Also, I did try not setting the password and pasting it in at the prompt, but that did not make a difference. — Will Turner

Daniel Mikusa Daniel Mikusa · Accepted Answer · 2018-08-21T02:56:08

You might be running into a bug in Cloud Controller. You can see the story here.

https://www.pivotaltracker.com/n/projects/966314/stories/159373079

The AWS ECR credentials are quite long, and can get truncated in the CCDB. Then when they're used to fetch your image they fail because they're incorrect.

There doesn't appear to be a fix as I'm writing this, but I'd expect one soon. Then you'd just need to upgrade your CF environment to a version that incorporates the fix. You can check the story which will be updated when it's complete.

Update - as of this update, there is a fix available in cf-deployment.

Use cf push with Amazon ECR?

1 Answers