Custom properties not saved correctly for Expando models in repeated StructuredProperty

Question

I am trying to use an Expando model as a repeated StructuredProperty in another model. Namely, I would like to add an indefinite number of Accounts to my User model. As Accounts can have different properties depending on their types (Accounts are references to social network accounts, and for example Twitter requires more information than Facebook for its OAuth process), I have designed my Account model as an Expando. I've added all basic information in the model definition, but I plan to add custom properties for specific social networks (e.g., a specific access_token_secret property for Twitter).

1/ Can you confirm the following design (Expando in repeated StructuredProperty) should work?

class Account(ndb.Expando):
    account_type = ndb.StringProperty(required=True, choices=['fb', 'tw', 'li'])
    account_id = ndb.StringProperty()
    state = ndb.StringProperty()
    access_token = ndb.StringProperty()

class HUser(User):
    email = ndb.StringProperty(required=True, validator=validate_email)
    created = ndb.DateTimeProperty(auto_now_add=True)
    accounts = ndb.StructuredProperty(Account, repeated=True)

2/ Now the problem I am facing is: when I add a Facebook account to my HUser instance, everything works fine ; however the problem rises when I append a Twitter account to that same instance, and add a new property not declared in the model, like that:

for account in huser.accounts:
    if account.state == "state_we_re_looking_for" and account.account_type == 'tw':
        # we found the appropriate Twitter account reference
        account.access_token_secret = "..." # store the access token secret fetched from Twitter API
        huser.put() # save to the Datastore
        break

This operation is supposed to save the access token secret in the Twitter Account instance of my User, but in fact it saves it in the Facebook Account instance (at index 0)!

What am I doing wrong?

Thanks.

Romain Romain · Accepted Answer · 2014-03-05T16:07:42

From what I understand, it seems like App Engine NDB does not support Expando entities containing Expando entities themselves.

One thing that I didn't realize at first is that my HUser model inherits from Google's User class, which is precisely an Expando model!

So without even knowing it, I was trying to put a repeated StructuredProperty of Expando objects inside another Expando, which seemingly is not supported (I didn't find anything clearly written on this limitation, however).

The solution is to design the data model in a different way. I put my Account objects in a separate entity kind (and this time, they are truly Expando objects!), and I've added a KeyProperty to reference the HUser entity. This involves more read/write ops, but the code is actually much simpler to read now...

I'll mark my own question as answered, unless someone has another interesting input regarding the limitation found here.

Custom properties not saved correctly for Expando models in repeated StructuredProperty

2 Answers