Error while exporting data from dynamodb to s3 using Amazon data pipeline

1
votes

I'm trying to use Data pipeline service to export data from dynamodb to S3 but keep getting the following error.

Unable to create resource for @EmrClusterForBackup_ due to: The supplied ami version is invalid. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID: xxx)

I have tried changing AMI to a lot of different values according to their documentation like all the ones in 4.x.x and 5.x.x series but keep getting the same error. I see there have been few threads but none have an answer. Hoping to find someone who's been able to solve this.

1
amazon-web-servicesamazon-s3amazon-dynamodbamazon-data-pipeline
I hope you're adding the AMI id given to you - Aakash Verma
@AakashVerma Can you specify what given means? By default, the AMI was 3.9 which didn't seem to work and hence I tried these different combinations. - Navjot Singh
Please post the connection command where you're facing this error. - Aakash Verma
In the event that this is a backup process, it may be relevant that backup and restore is now offered natively through DynamoDB backup - F_SO_K
I tried using the backup option but apparently, it's not yet available for my account. All I see is 'this feature will be coming to your account'. So I used Hive to do the job. - Navjot Singh

1 Answers

4
votes

Check if region of your EMR Cluster is the same as region of pipeline. There is a bug and by default it is set as region of DynamoDB (which might be different)