Cloud datastore client.GetAll type mismatch but client.Get works

Question

I am writing a go application in app engine that connects and returns some info from datastore entities. I'm having an issue where client.Get is working with a predefined struct but client.GetAll is throwing a type mismatch (or vice versa).

I am using the following struct with both:

type myStruct struct {
    ID              int64
    Field1          string
    Field2          string
    Release_Date    time.Time
}

This works when Release_Date is defined as time.Time (fails if int):

k := db.datastoreKey(id)
myStruct := &myStruct{}
if err := db.client.Get(ctx, k, myStruct ); err != nil {
        return nil, fmt.Errorf("datastore: %v", err)
}

func (db *datastoreDB) datastoreKey(id int64) *datastore.Key {
    return datastore.IDKey("myEntityType", id, nil)
}

This fails when Release_Date is defined as time.Time (works if int):

var myStructs []*myStruct 
q := datastore.NewQuery("myEntityType").
            Project("field1", "field2", "release_date").
            Order("field1")
keys, err := db.client.GetAll(ctx, q, &myStructs)
if err != nil {
        return nil, fmt.Errorf("datastore: %v", err)
}

The error:

datastore: cannot load field "release_date" into a "myStruct": type mismatch: int versus time.Time

(or vice versa when I swap the Release_Date definition).

Any ideas what is wrong or is this a bug?

Additional info:

I've looked at the entity dashboard and there it lists the data type as Data/Time and when I retrieve the entity with Get it loads fine into the struct and I can use the object in my code.

I have no idea why this isn't working.

Are you sure that every record has a release_date that is a time.Time? that seems like the most likely cause - there actually is a record (or records) with an int for that field, and there actually are records with a time.Time, so no matter which one you choose, when you do GetAll, it fails, since it will find at least one that doesn't match. — dave
I did check (I'm only testing with three entities right now) and just double-checked to be sure. All are the same data type. — tmwoods
There's not enough information here to help -- it still looks like either @dave is right about the entities not all having been stored in the same way, or possibly that you have two different entity types. We don't see how you create your key (k) in the Get example, so we don't know if that's using the same "myEntityType" as the GetAll example. One way or another, though, we know from your error messages that you've stored some entities' Release_Date as int and some as time.Time. — Darshan Rivka Whittle
Also, I've never seen NewQuery, Project, or Order with lower-case parameters, since they only work with exported types and fields. I'm not sure if the datastore package just treats them case-insensitively (since these are just string parameters referring to things you've already stored, that's entirely possible), but it stands at to me as unusual, and incongruous. — Darshan Rivka Whittle
I added where I create k. I also nuked all entries in my datastore and added a single entry (fun fact, it means you have to rebuild the indexes) triple-checking the data types. The parameters you refer to are the named fields in my datastore so they should match case, no? — tmwoods

tmwoods tmwoods · Accepted Answer · 2018-05-10T15:55:33

So this isn't an elegant solution, but it's what worked for me.

What I think was the cause is that the very first entity I created had a field Release_Date of a type int but I later added entities using the type Date/Time. During my testing I had ended up removing the entity with type int for Release_Date so the only remaining entities had Date/Time. Despite having no entities with type int for Release_Date, my code failed server-side. It kept expecting that field to be of type int.

My theory is that the entity definition was established when I first created the entity and was not updated when I changed the data type later on even though no entities remained using type int.

So what I ended up doing was backing up my entities, removing ALL of them, then re-adding the entities with only Date/Time in the Release_Date field. Once I did that, everything worked fine. I still have the casing the same as shown above (i.e. my Struct uses Field but my query uses field).

This would likely be an easy thing to reproduce but I haven't gotten around to it. If anyone else gets around to testing please add a comment, otherwise I will eventually and submit a bug report.

Cloud datastore client.GetAll type mismatch but client.Get works

1 Answers