Need Pattern for lookup tables in Google App Engine

2
votes

I'm using the Python version of Google App Engine and Datastore. What is a good way to load a table that will contain lookup data?

By look up data I mean that after the initial load no rows will need to be inserted, deleted, or updated

Blowing away all rows and reloading the table is not acceptable if it destroys referential integrity with other rows referring to it.

Here is an example of a couple kinds that I am using that I want to load lookup data into

class Badge(db.Model):
    name = db.StringProperty()
    level = db.IntegerProperty()

class Achievement(db.Model):
    name = db.StringProperty()
    level = db.IntegerProperty()
    badge = db.ReferenceProperty(reference_class=Badge)

Here is an example of a kind not holding look up data but referring to it

class CamperAchievement(db.Model):
    camper = db.ReferenceProperty(reference_class=Camper)
    achievement = db.ReferenceProperty(reference_class=Achievement)
    session = db.ReferenceProperty(reference_class=Session)
    passed = db.BooleanProperty(default=True)

I'm looking to find out two things:
What should the code to load the data look like?
What should trigger the loading code to execute?

2
pythongoogle-app-enginegoogle-cloud-datastore
your two questions are very ambiguous. "What should the code to load the data look like?" It's difficult to say. The structure of your models doesn't make it evident exactly what you are trying to accomplish or what business rules may apply. "What should trigger the loading code to execute?" Absolutely no idea what this mean. - Adam Crossland
The key point here is that it is lookup data. It's the database equivalent of a programming constant. The data is defined once and then never changed. It is loaded by the app either inside python or in a .yaml file. It is not effected by user input. - zudduz

2 Answers

2
votes

If it's really created once and never changes within the lifetime of a deployment, and it's relatively small (a few megs or less), store it with your app as data files. Have the app load the data into memory initially, and cache it there.

0
votes

I think you're referring to some kind of write-behind cache but it is not really that clear from your question.

This is a common pattern for reading data slow a slow source (like a database/disk) and caching it in a fast source (like memcache/memory) for quick retrieval later. It is then your responsibility to empty out all the cached items when things change.

See the first example on Using memcache