How to change google cloud datastore kind name ?

Question

We are using google cloud datastore to store data and unfortunately initially haven't followed name convention for naming kinds and now we want to change names of already existing kinds in datastore.

We have already accumulated a lot of data and a lot of computation was involved generating that data so populating that complete data again just to rename a kind isn't an option for us.

Have tried finding it out but with no luck. So is there something which I have missed and can be helpful in achieving it ?

Answer 1

You cannot rename a kind in datastore, Because the kind name is embedded with the key of each element in the datastore. You will have to write a script to migrate all the data to the new kind.

Answer 2

If you're only trying to keep your code clean, you could alias your models in code as a simple solution. When using GQL you'll still need to reference your kinds by their original name though.

As @Akash Dathan has already said, you can migrate the data by scripting it. I'll provide some more detail:

GQL can't be used for this kind of migration. See the following doc (for Python), which states that 'GQL is a SQL-like language for retrieving entities and keys':

https://cloud.google.com/appengine/docs/standard/python/datastore/gqlreference?csw=1

Python

For a practical Python example, reference here from slide 29 'Renaming Models:

https://www.slideshare.net/RyanMorlok/data-migrations-in-the-app-engine-datastore

Java

If you're using Java you probably want to look at the Java DataStore API -

https://cloud.google.com/appengine/docs/standard/java/datastore/api-overview#Java_Kinds_keys_and_identifiers

If your entity management is done using some kind of unit of work pattern that's generically implemented (which is fairly common), then the basic migration code using the Java DataStore API, may look something like:

    List<MyKind1> allKind1Entities = myDataService.GetAll(MyKind1);

    for (MyKind1 myKind1: allKind1Entities) {

        MyKind2 myKind2entity = MyKind1.MapToMyKind2(myKind1);
        try {
            long myKind2entity Id = myDataService.Add(myKind2entity);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    myDataService.PurgeAll(MyKind1);

If you're doing any manual linking between entities through another 'kind' that contains linking entities, then it gets more complex as you'll need to update your ID references (since the new myKind2 entities will all have new id's)

Updating the schema in Production

If your app is running under app engine and you need to update the schema live , you'll find more details here about considerations when updating a model in code - https://cloud.google.com/appengine/articles/update_schema

Here's a very good YouTube from Google on how to do live migration without downtime - https://youtube.com/watch?v=qFU5aTT1Eqk