How to handle querying across remote and cache data

1
votes

Looking to find a good way to query across remote data and local cache simultaneously. I need to be able to query remote data but merge the results with the local changes.

So if I add an item "Bob" and I get the first 5 results alphabetically ordered ascending, Instead of receiving "Aaron, Adam, Alice, Ashley, Bud", I would instead receive "Aaron, Adam, Alice, Ashley, Bob" and if I queried for the second 5 results (skip 5, take 5) the list would start with "Bud". This also needs to handle deleting, so if I have locally deleted items 1 and 3 and I query for the first 50 items, I want to receive items 2 and 4 through 52. And lastly queries for filtering need to address both the server and the local cache changes: so if I change "Sam"'s name to "Bob" and query for all names beginning with B, I should get Sam's new Bob record with Bill and the guys.

Normally we would just download all the rows with the initial query and allow for queryLocally to handle the work, but there is an issue because we are working with 10,000 records or more.

Any help is appreciated!

3
breeze
Frankly this is a very difficult problem ... much trickier than it seems at first. We solved it in DevForce with the EntityQueryPager. If you (and others) want this feature, please vote for it on User Voice. - Ward
Ward, my vote is in. Thank you for your response. This is exactly the solution that we require. - Jonneh
I don't want to raise your hopes too high. I'm glad you're voting for it. If it gets enough votes, we'll move it up the queue. We'd be happy to provide source of the DF component if someone wants to author a JS version before we can get to it. Let us know. - Ward

3 Answers

1
votes

One possible solution to your scenario would be to query the server for "a little bit more" then expected as a result - but eventually always filter the local cache. Make the local cache the only source and filter/take/skip from it.

Now let's try to define what means "a little bit more".

if I change "Sam"'s name to "Bob" and query for all names beginning with B, I should get Sam's new Bob record with Bill and the guys.

This should just work as you expect - no need to fetch more then "names beginning with B" from the server.

if I add an item "Bob" and I get the first 5 results alphabetically ordered ascending, Instead of receiving "Aaron, Adam, Alice, Ashley, Bud", I would instead receive "Aaron, Adam, Alice, Ashley, Bob" and if I queried for the second 5 results (skip 5, take 5) the list would start with "Bud".

For this to work you will need to fetch remotely like this: take numberOfEntitiesChangedLocally + ( pageNr * recordsPerPage ) and then skip and take from the local cache locally in the usual way.

0
votes

There are three examples of combined remote + local queries located in the \Samples\DocCode. Perhaps they will help.

Comments added here. The full code is in queryTests.js

1. * Combine remote and local query to get all customers * including new, unsaved customers * v1 - Using FetchStrategy

2. * Combine remote and local query to get all customers * including new, unsaved customers * v1=using FetchStrategy.FromLocalCache

3. * Combine remote and local query to get all customers * including new, unsaved customers * v2=using ExecuteLocally()

An an interesting example of how not to do it anymore might interest you as well (also in queryTests:

    /*********************************************************************
    * This portion of the "queryTests (by id)" module  
    * tests a hand-built async getById utility that was the way to do it 
    * before EntityManager.fetchEntityByKey
    * A curiosity now.
    ********************************************************************/

    // This hand-built async getById utility method returns a promise.
    // A successful promise returns the entity if found in cache 
    // or if found remotely.
    // Returns null if not found or if found in cache but is marked deleted.
    // Caller should check for query failure.
    // 'queryResult' reports if queried the remote service 
    // and holds a found entity even if it is marked for deletion.
    // 
    // This fnc has been replaced by EntityManager.getEntityByKey.
0
votes

This must be achievable in java script, silver light is able to do that somebody might need to see how it is done is silver light and port it to java script.

I think logic should be when a switch is set(AlwaysRefreshDataOnQuery =false), then skip,overwriting the rows that has been modified(including deletes) otherwise simply clear the local cache and refresh it with new Data.

Silver light has been very powerful platform, i have not seen any platform to date as powerful as silver light(at least in .NET world).