AWS DMS (Data Migration Service) Table Full Load Timeout after 5 minutes

Question

I'm trying to use DMS to migrate a MySQL database from EC2 (200gb) to a new RDS Aurora instance. The problem is that not all rows are migrated.

Every table only get 5 minutes to fully load the table, and if it's not finished after these 5 minutes, DMS stops loading this table and goes into "Table completed" state.

I find this in the UNLOAD SOURCE DETAILED_DEBUG logfile:

Entering mysql_unload_init_table, unloadtimeout: 300 (mysql_endpoint_unload.c:131)

Does anybody know how to increase this? I think there was once the possibility to call aws dms modify-endpoint api and pass MySQLSettings but this is not available anymore. How can this be achieved now?

I have a table with 200.000.000 records to migrate and this takes obviously more than 5 minutes.

Thanks for your help!

Answer 1

I get the answer on a different channel. This might help someone else in the future!

We were encountering the same issue doing a MySQL RDS -> MySQL Aurora migration with DMS. Any table that took longer than 5 minutes would stop migrating, claim to be complete, then fail validation. Adding the following source connection attributes fixed the issue for us (set UnloadTimeout to a relevant value for you, I'm using 1 day as a test):

ResumeFetchForXRows=0;UnloadTimeout=86400

Additionally, AWS support sent me the following:

"There is a known issue when doing full load on MySQL Source,MySQL server will return there isn't more data when fetching the next records even if there are more records still in the table. To resolve the issue, I recommend you to add "Resumefetchforxrows=0" into extra connection attribute of the source endpoint. This should fix your issue. To do this edit your source endpoint in the DMS console and add the attribute."

I found that the UnloadTimeout parameter had the greatest impact for us.