Requirement: Restore DynamoDB table from S3 Backup location.
We created Data Pipeline job, and then edit Resources section in Architect Wizard.
We placed 20 instances under Core Instance count, but after the Data Pipeline job activation, EMR Cluster was created with only one master and one core instance.
Could you please suggest us, how to increase the number of cores under Resources section
You might be hitting the total EC2 resource limit. You can have only 20 On-Demand EC2 instances running unless you have requested a limit raise. Datapipeline acknowledges this limit. You have to increase the EC2 Ondemand resource limit using the link below.
A similar scenario from the docs:
If you configure AWS Data Pipeline to automatically create a 20-node Amazon EMR cluster to process data and your AWS account has an EC2 instance limit set to 20, you may inadvertently exhaust your available backfill resources. As a result, consider these resource restrictions in your design or increase your account limits accordingly.
If you require additional capacity, you can use the Amazon Web Services Support Center request form to increase your capacity.
