Integrating GAE Search API with Datatstore

0
votes

When a document is stored into both the Cloud datastore and a Search index, is it possible when querying from the Search index, rather than returning the Search index documents, returning each corresponding entity from the Cloud datastore instead? In other words, I essentially want my search query to return what a datastore query would return.

More background: When I create an entity in the datastore, I pass the entity id, name, and description parameters. A search document is built so that its doc id is the same as the corresponding entity id. The goal is to create a front-end search implementation that will utilize the full-text search api to retrieve all relevant documents based on the text query. However, I want to return all details of that document, which is stored in the datastore entity.

Would the only way to do this be to create a key for each search doc_id returned from the query, and then use get_multi(keys) to retrieve all relevant datastore entities?

2
pythongoogle-app-enginegoogle-cloud-datastoregoogle-search-api
I guess an alternative to converting to pure Search documents is to rely only on datastore queries and recreate full-text-search type capabilities by building multiple indexes. Hoping someone has more insightful thoughts than these options. – yoonjesung
You need to be more specific with your question, it is unclear what you want answered. – danielx

2 Answers

1
votes

There is no first class support for this, your best bet is to make the document id match the datastore key and route all put/get/search requests through a single DAO/repository tier to ensure some level of consistency.

You can use parallel Async writes to keep latency down, but there's not much you can do about search not participating in transactions. It also has no defined consistency, so assume it is eventual, and probably much slower than datastore index propagation.

0
votes

You can store any information that you need in the Search API documents, in addition to their text content.

This will allow you to retrieve all data in one call at the expense of, possibly, storing some duplicate information both in the Search API documents and in the Datastore entities. Obviously, having duplicate data is not ideal, but it may be a good option for rarely changing data (e.g. document timestamp, author ID, title, etc.) as it can offer a significant performance boost.