App Engine Datastore query result is returned randomly

Question

As an exercise to better understand how the Datastore works, I have created within the HttpServlet class of my project a datastore with several entities and relative properties. After creating and populating the datastore, I immediately query it in order to return a Json object back to the Client so it can dynamically update its UI.

It works, but I have a problem: after the first time I query the datastore, I always receive a result ordered differently, often with duplicates of the same items.

This is my HttpServlet code:

public class MyServlet extends HttpServlet {
    ArrayList<Tour> m_tours = new ArrayList<Tour>();
    Key tourKey;
    DatastoreService datastore;
    @Override
    public void doGet(HttpServletRequest req, HttpServletResponse resp)
        throws IOException {
        //nothing special here
    }

    @Override
    public void doPost(HttpServletRequest req, HttpServletResponse resp) throws IOException {
        populateDatastore();

        String asyncMessage = req.getParameter("order");
        if(asyncMessage.equals("tours")){
            m_tours = getTours();
        }
        if(asyncMessage.equals("selectTour")){

        }

        Tours tours = new Tours(m_tours);
        resp.setContentType("application/json");
        PrintWriter out = resp.getWriter();
        out.print(new Gson().toJson(tours));
        out.flush();
    }
    private DatastoreService populateDatastore(){

    datastore = DatastoreServiceFactory.getDatastoreService();
    tourKey = KeyFactory.createKey("availabletours", "tours"); //parent

    Entity tour = new Entity("tour", tourKey);
    tour.setProperty("tourname", "Tour0");
    tour.setProperty("tourinfo", "info0");
    datastore.put(tour);

    Entity tour1 = new Entity("tour1", tourKey);
    tour1.setProperty("tourname", "Tour 1");
    tour1.setProperty("tourinfo", "info 1");
    datastore.put(tour1);

    //..... and so on

    return datastore;
}

private ArrayList<Tour> getTours(){
    ArrayList<Tour> toursArray = new ArrayList<Tour>();

    Query query = new Query(tourKey);
    List<Entity> tourss = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(9));
    for (Entity t : tourss) {
        String tourname=t.getProperty("tourname").toString();
        String tourinfo=t.getProperty("tourinfo").toString();
        Tour ts = new Tour(tourname,tourinfo);
        toursArray.add(ts);
    }

    return toursArray;
    }
}

What am I doing wrong? What is the best way to manage a datastore?

* EDIT *

After uniforming the Entity objects to the same type and setting an ancestor query, as suggested by @Jeff Deskins:

    datastore = DatastoreServiceFactory.getDatastoreService();
    tourKey = KeyFactory.createKey("availabletours", "tours"); //parent

    Entity tour = new Entity("tour", tourKey);
    tour.setProperty("tourname", "Tour0");
    tour.setProperty("tourinfo", "info0");
    datastore.put(tour);

    Entity tour1 = new Entity("tour", tourKey);
    tour1.setProperty("tourname", "Tour 1");
    tour1.setProperty("tourinfo", "info 1");
    datastore.put(tour1);

    //... ...  same way with the others

I set an ancestor query:

    Query query = new Query("tour").setAncestor(tourKey);
    List<Entity> tourss = datastore.prepare(query).asList(FetchOptions.Builder.withLimit(9));
    for (Entity t : tourss) {
        String tourname=t.getProperty("tourname").toString();
        String tourinfo=t.getProperty("tourinfo").toString();
        Tour ts = new Tour(tourname,tourinfo);
        toursArray.add(ts);
    }

And this is the result I get:

First time: first query Second time: second query

EDIT 2: Answer - Apart from performing an ancestor query, I also needed to apply a Sort to the Query. Therefore the solution is to add a property to each entity:

  Entity tour = new Entity("tour", tourKey);
        tour.setProperty("tourname", "Tour 1");
        tour.setProperty("tourinfo", "info 1");
        tour.setProperty("order","0");
  Entity tour = new Entity("tour", tourKey);
        tour.setProperty("tourname", "Tour 2");
        tour.setProperty("tourinfo", "info 2");
        tour.setProperty("order","1");

and then apply a sort filter to the query:

    ArrayList<Tour> toursArray = new ArrayList<Tour>();

    Query query = new Query("tour").setAncestor(tourKey).addSort("order");
    List<Entity> tourss = new ArrayList<Entity>();
    tourss = datastore.prepare(query).asList(FetchOptions.Builder.withDefaults());

    for (Entity t : tourss) {
        String tourname=t.getProperty("tourname").toString();
        String tourinfo=t.getProperty("tourinfo").toString();
        Tour ts = new Tour(tourname,tourinfo);
        toursArray.add(ts);
    }

now the result is sorted properly

you'll have to be more precise in what results you get. The datastore IS eventually consistent, so it's normal that if you query right after adding data, it isn't there right away. — Patrice
It looks like you're doing an ancestor query, so you should get strong consistency. Can you give an example of what you're seeing? — tx802
See my edit. I have also noticed that when I set the FetchOptions.Builder.withLimit(9) to 20 I get 20 randomized items with lots of repetition. — Cris

Jeff Deskins Jeff Deskins · Accepted Answer · 2015-03-31T20:00:48

Your Entity objects should follow the same pattern if they are all the same type.

Creating different entities of same type with same parent key:

Entity tour1 = new Entity("Tour", tourKey);
Entity tour2 = new Entity("Tour", tourKey);

After populating and saving the above objects to the datastore, you can then query by:

Query tourQuery = new Query("Tour")
                         .setAncestor(tourKey);

The ancestor query provides strong consistency. https://cloud.google.com/appengine/docs/java/datastore/queries#Java_Ancestor_queries

App Engine Datastore query result is returned randomly

1 Answers