How to reject concurrent duplicate saves using Objectify for AppEngine app

Question

We have an entity defined as (id, name, school, & OTHER-FIELDS). The id is auto assigned by datastore. And we want to make sure the combination of <name, school> is unique in datastore.

Our code write like this:

/* First check uniqueness, then save */
public void savePerson(final Person newPerson) {
    List<Person> existing = ofy().load().type(Person.clas)
            .filter("name=", name)
            .filter("school=", school)
            .list();
    if (existing == null || existing.isEmpty()) {
        ofy.transact(new VoidWork() {
            @Override
            public void vrun() {
                ofy.save.entity(newPerson).now();
            }
        }
    }
}

But it still happens if multiple same savePerson calls happen at the same time (unpreventably we will face this), the duplicate check will fail (I think because when they start call savePerson() the data is not actually saved), and we find in AppEngine datastore it ended up with duplicate entities with the same <name, school> combinations.

We really want to make sure in our datastore given <name, school> we will end up with a unique entity Can someone teach me how to add duplicate check atomically? Does it help if I move the existing entity check code into the transact context?

For example will the following code work?

/* Put filtering search into transaction too */
public void savePerson(final Person newPerson) {
    ofy.transact(new VoidWork() {
        @Override
        public void vrun() {
            List<Person> existing = ofy().load().type(Person.class)
                    .filter("name=", name)
                    .filter("school=", school)
                    .list();
            if (existing == null || existing.isEmpty()) {
                ofy.save.entity(newPerson).now();
            }
        }
    }
}

Let me know if I should provide more details.

It would require some re-structuring of your database design, but would a unique identifier at application level be acceptable? That would perform better than the higher overhead of a uniqueness scope on a Person kind. See this recent StackOverflow discussion. — Martin Berends

Alejandro C. Alejandro C. · Accepted Answer · 2014-08-27T16:15:33

For doing what you want, it's compulsory to use a single transaction to both read if there's already the given name + school and save the entity; otherwise, if you have many concurrent users, you will end having duplicates. Instead of loading a whole list, you can just load the first entity [i.e. first()]. Make sure that 'name' and 'school' are indexed (i.e. @Index). Also check that you have ObjectifyFilter in web.xml.

However notice that it is possible to be two different persons with the same name in the same school, so you should add another way to differentiate persons.

How to reject concurrent duplicate saves using Objectify for AppEngine app

1 Answers