Objectify - AppEngine Alternatives

Question

we have a very big app off objectify but we are finding lot of issues with the underlying platform. Eg: inequality filters more than 1 cannot be used, sorting based on custom data (end user variables) are two very big pains. Ours is a vertical CRM.

Since we are using objectify, is there a way to move to a different cloud platform without lot of pain?

We are ok to restructure some of the application for it but what would you recommend us to move to assuming that AppEngine is not working out? Does objectify work or can be made to work for other platforms easily?

Thanks.

Answer 1

MongoDB's Morphia interface was originally based off of an early version of Objectify. As key/value stores go, MongoDB will seem fairly familiar to GAE users. If you're looking for a low-pain transition, this is probably the best option.

Note that some things (like ad-hoc and less-than-optimal queries) are easier with MongoDB, but some things (like transactions) are harder. And of course make sure that it fits your scaling requirements.

Answer 2

If you're staying on App Engine, the Java Datastore API page suggests Slim3, also the low-level Datastore API, along with Objectify.

It also mentions JDO and JPA, but the reason to use those is compatibility with other implementations of the same interfaces, and these are subset implementations so the compatibility is limited.

The Storing Data in Java page lists other choices Google Cloud SQL and Google Cloud Storage. These will require bigger changes to your code.

Answer 3

One inequality filter is an App Engine's limitation, not Objectify's. It was never a problem for us, and we have a very large and complex application with close to 100 entity kinds. There are many tips across SO on how to work around this limitation.

I have no idea why you have a problem with sorting data. As long as it is indexed, you can sort based on any criteria - user-generated or not. Maybe you can elaborate on this, and we can help with a solution.

App Engine is a very different platform, and it may take time for folks who are used to only relational databases to learn and appreciate its power. What you see as limitations are actually features - one inequality filter, for example, allows for very fast (and consistent) query times regardless of the number of entities. I would even state that the larger the application, the more it will benefit from using a non-relational (App Engine) Datastore.

It will probably take you less time to learn how to use App Engine than to move to an entirely different platform.

UPDATE:

I would suggest the following approach to your plant question (based on a very limited information I have about your data model and usage patterns).

1. If users can add any number of custom properties to an object, the ultimately flexible solution is to store all of them as unindexed properties in the datastore. Then when a user initiates a query, you retrieve all entities of a given kind and filter them in your code as opposed to relying on Datastore indexes. It may seem like an inefficient approach, but it is dramatically cheaper to add and retrieve entities with few indexed properties than entities with many indexed properties. Overall, your total costs may end up being lower even though you over-read on each custom query, unless your users add >10k entities of the same kind. You can even estimate this in advance as the Datastore shows the number of write operations on each entity in the Datastore viewer.

If you have a number of standard (non-user generated) indexed properties on these entities, you can further optimize this by splitting each entity into a standard part (e.g. "Plant") and custom part ("Plant_custom") with end-user generated properties. This way an update to the custom part will be much cheaper as it won't trigger index re-writes on standard properties. Then, if a query involves standard properties, you retrieve a list of entities that match search criteria on standard properties using a cheap keys-only query, then retrieve custom entities using a cheap get operation (standard and custom entities should have the same ID), and finally filter based on custom search criteria.

2. A totally different solution is to use App Engine Search API for your custom properties. It is much more flexible that the Datastore, but you need to compare costs. Still, it may be a cheaper solution than indexing every custom property in the Datastore.