Exchanging NDB Entities between two GAE web apps using URL Fetch

Question

I am planning to exchange NDB Entities between two GAE web apps using URL Fetch.

One Web app can initiate the HTTP POST Request with the entity model name, starting entity index number and number of entities to be fetched. Each entity would have an index number which would be incremented sequentially for new entities.

To Send an Entity:
Some delimiter could be added to separate different entities as well as to separate properties of an entity. The HTTP Response would have a variable (say "content") containing the entity data.

Receiving Side Web APP:
The receiver web app would parse the received data and store the entities and their property values by creating new entities and "put"ting them

Both the web apps are running GAE Python and have the same models.

My Questions: Is there any disadvantage with the above method? Is there a better way to achieve this in automated way in code?

I intend to implement this for some kind of infrequent data backup design implementation

I would have to ask why you are running two independent applications that are intended to share data. Why not run both on the same application. — Tim Hoffman
@TimHoffman I would be frank.. I want to have the data entry staff go ahead and feed data to this "data app". This insulates the "main app" from poor design issues in future when I might have to redesign the Models(database design). In this way, I could just rewrite the "main app" and I would have all the data ready to be fed to it. My "main app" stores some of the "data app" data in some specialized data (Models) which are designed to give better query response time.. If you feel this approach is not right, pl feel free to tell me — gsinha
I think this is recipe for disaster IMO, not adhering to DRY, you now have to manage data in two locations, keep things synchronized. At which point you now need to start versioning entities, or how do you know which one is correct. Your different versions (modules) for admin vs normal users. Develop a good security model. I don't see how this approach insulates you from major changes if you need to change you data structure in any significant way. Either way you will need to perform data migration. All it takes is the models to drift a tiny bit - and then your in a world of hurt. — Tim Hoffman
@TimHoffman Thanks for your response. Since data changes would not be frequent (like once in three months for one entity or even later), and whenever they happen, they would be first done in "data app" from where we could push them to the "main app".. This would insulate me from design changes in my "main app" as I can create the entities as per my new model after receiving data(when I do design change) from "data app" — gsinha
If you plan do to this, I would look at using the remote_api, rather than POST/ this means you can write directly to the remote datastore with the models. Also think about id allocation. You need to create all keys so that references (KeyProperties) point to the entities. — Tim Hoffman

voscausa voscausa · Accepted Answer · 2014-04-05T11:56:18

You can use the NDB to_dict() method for an entity and use json to exchange te data. If it is a lot of data you can use a cursor.

To exchange the entity keys, you can add the safe key to the dict.

Exchanging NDB Entities between two GAE web apps using URL Fetch

2 Answers