Copying Google Gloud Datastore entities from one namespace to an other

Question

I'm copying Google Cloud Datastore entities from one namespace to an other with java like this:

Key newKey = Key.newBuilder(oldEntity.getKey()).setNamespace(NEW_NAMESPACE).build();
datastore.put(Entity.newBuilder(oldEntity).setKey(newKey).build());

Since the entities have numerid id's generated by Datastore and the id's of the copied entities need to remain the same I need to also let Datastore know to allocate these id's, for that I'm using DatastoreService.allocateIdRange

But this is giving me an error:

Exceeded maximum allocated IDs

Does this mean that there is no way to achieve what I'm trying to achieve?

EDIT 1

The code that allocates the ids:

@POST
public void post(
        @QueryParam("namespace") String namespace,
        @QueryParam("kind") String kind,
        @QueryParam("id") long id,
        @QueryParam("parentKind") String parentKind,
        @QueryParam("parentName") String parentName,
        @QueryParam("parentId") Long parentId
) {
    NamespaceManager.set(namespace);
    DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
    Key parent = null;
    if (parentKind != null) {
        if (parentName != null)
            parent = KeyFactory.createKey(parentKind, parentName);
        else
            parent = KeyFactory.createKey(parentKind, parentId);
    }
    ds.allocateIdRange(new KeyRange(parent, kind, id, id));
}

Is it possible that you exceeded the limit of 500 allocated IDs per operation? — Eien
@Heigo Can we see the code where you call allocateIdRange? If you're calling it in a loop or something you could actually have allocated all of the IDs. — Venantius
I know the same key IDs can be used with a different key ancestry (i.e. different entity groups), but docs don't appear to mention if the same is true for just a different namespace. If it's not true that most likely explains the error. — Dan Cornilescu
@Venantius I'm doing this in a separate request, since I have to be able to access it from outside of AppEngine. So this can not be the issue. — Heigo

Raghuram Kasyap Raghuram Kasyap · Accepted Answer · 2018-04-28T05:08:18

I think you can skip the key creation. This is my code for updating namespaces:

GoogleCloudDatastore gds = new GoogleCloudDatastore();
Datastore ds = gds.getCredentials();

final KeyFactory keyFactory = ds.newKeyFactory().setKind(entityKind).setNamespace(NAME_SPACE);
Key newKey = ds.allocateId(oldEntity.getKey);
Entity newEntity= Entity.newBuilder(newKey).set("someParameterName", someParameterValue).build();

Hope this helps.

Copying Google Gloud Datastore entities from one namespace to an other

2 Answers