Using the appengine datastore admin to move data to a new application - how long should this take?

1
votes

I am moving to the high-replication datastore which is relatively simple when using the AppEngine Datastore Admin feature to copy all data from the old application to the high-replication application.

However, I have several applications (and started by copying applications with smaller databases), and have seen that the rate of copy is about 1Gb every two hours. This is acceptable for some of my applications, however I am now going to be moving an application with about 14GB of data, and would prefer to not have to shut down my website for more than a day while transferring the data.

Is there any way to make this process go faster? Are these data transfer speeds typical for the Datastore Admin tool?

Kind Regards

1
google-app-enginecopyadmindelaydatastore
I forgot to mention (for anyone considering doing such a transfer) the CPU usage for these kind of transfers costs about $7 per GB. Therefore, to transfer a 14GB application will cost about $100 ... – Alexander Marquardt
I just realized that I was only counting the $7 per GB on the receiving application, but the sending application also costs about $4.50 per GB to send the data (CPU usage). Therefore, this migration will cost about $160. I am not complaining about the pricing, but thought it might be relevant to people considering moving the the high replication datastore. – Alexander Marquardt

1 Answers

2
votes

Google are working on a new migration tool, which will allow a shorter read-only period while migrating.

They mentioned it when 1.5.3 was released:

http://googleappengine.blogspot.com/2011/08/app-engine-153-sdk-released.html

You can try signing up for early access to the migration tool here:

https://docs.google.com/spreadsheet/viewform?authkey=CLXR0LMN&formkey=dERMcDZuMnlycHoyZDd4Vy1PNXlhWlE6MQ&ndplr=1