Export more than 50 Kinds from Google Cloud Datastore

1
votes

We are attempting to use the new datastore export to export more than 50 different entity kinds. Currently this is not possible in a single backup request. Ideally the backups would all end up in the same Cloud Storage bucket (split by kind).

Things we've tried already:

  • Exporting everything (not specifying entity kinds to filter by): This process takes over 4 hours to complete and results in all entity kinds being combined together in a single folder rather than one folder by kind.

  • Splitting the export into multiple chunks: When we try and export multiple different entities into the same bucket, we get the error Path already exists: /BUCKET_NAME/2018-08-21T09:37:53/2018-08-21T09:37:53.overall_export_metadata

1
google-app-enginegoogle-cloud-datastore
Did you try using a different, per-kind namespace path as per Set up your environment: BUCKET="YOUR_BUCKET_NAME[/NAMESPACE_PATH]" - Dan Cornilescu
Thanks for your comment. Will give that a try now. - drei01
@drei01, did that solution work for you? - A.Queue
This suggested solution didn't work. We asked Google to increase the limit in the end. - drei01

1 Answers

2
votes

The 50 kind limit is arbitrarily set by Google. We raised a support ticket with them and they were able to increase the limit for our account.