Get plain JavaScript objects from EntityQuery (for large result sets)?

Question

Breeze is brilliant but we're running into memory issues with large result sets. I have an idea of what I'd like to achieve and would value any thoughts about the best approach...

• For special (large) result sets I'd like to get Breeze to execute a query against a server service like normal but NOT turn those results into Entities - instead strip out the extra "Breeze" JSON and give me a plain list of JavaScript objects (ideally with the property casing changed to suit the client).

• I could then put these results into an ObservableArray and bind them using Knockout - but without the overhead of each of the properties being observable or tracked.

• when the user indicated that they wanted to edit a record, I could then create an appropriate Breeze entity and splice it into the ObservableArray in place of the plain Javascript object. As long as the property names matched exactly then the Knockout bindings shouldn't be any the wiser.

• Breeze could then track changes and provide saving like normal for the few records that actually get edited.

My hope to retain some of the value of Breeze but avoid the overhead for these large result sets. Specifically I want to avoid create custom (non-Breeze) server-side controller methods if possible.

Can anyone suggest the best places to break into Breeze to try and achieve this? (or a better approach)

Thanks for any comments.

Answer 1

Basically Breeze will only create 'observables' for any json that has an entity type described in metadata. When breeze parses the incoming results of any query it tries to 'recognize' any entityTypes returned within the json payload, if it cannot then it just returns the json objects in their raw form. This parsing is performed by a 'default' JsonResultsAdapter.

This means that if you don't tell Breeze about the 'entityTypes' in your large result sets, then you will get exactly the results that you want. Breeze gets these entityType 'definitions' during the initial metadata call to the server. For models involving the Entity Framework, these types are provided automatically for any type described in your model. This mean that any DTO's that you create will NOT have entityTypes unless you have explicitly provided metadata for them. So one option is to simply to return your large result sets as DTO's.

The other, possibly better alternative is if these large result sets DO contain entityTypes that breeze does know about, but you just want to ignore them for specific queries. In this case your best option is to write a custom JsonResultsAdapter for these queries that simply does not return an entityType. Something like this:

   var customJsonResultsAdapter = new breeze.JsonResultsAdapter({
        name: "ignoreEntityTypeAdapter",

        visitNode: function (node, mappingContext, nodeContext) {
            return {  };
        }
    });

    var query = EntityQuery.from("QueryThatResultsLargeResultSet")
         .where(...)
         .using(customJsonResultsAdapter);

Hope this was clear enough.

Answer 2

As per the comments above we missed having Breeze hook up all of the navigation properties, etc when using the customJsonResultsAdapter approach. Essentially we wanted the goodness of Breeze without the overhead of making all entity properties into Knockout observables (until we needed them to be).

EDIT: The answer has been changed to use two utility functions due to bugs in the previous custom modelLibrary approach

As an alternative solution we changed to using the built in Breeze backingStore modelLibrary adapter and added a couple of functions to make individual entities Knockout observable when required. The consequences of this are:

• This is a "lite" approach and is ideal for read-only data.
• Breeze will intercept changes to these properties and track them like normal.
• We can still bind the lite entities using Knockout but, as the properties aren't observable by default, the web page wont get updated if the property value changes.

When a user does want to edit an entity we do something like...

    function startEditing(entity, uiElement) {
        if (!breeze.utils.isObservable(entity)) {
            breeze.utils.makeObservable(entity);
            ko.applyBindings(entity, uiElement);
        }

A couple of important notes:

• we need to re-applyBindings after making the entity observable so that property changes trigger Knockout to update the UI.
• our makeObservable functions use Steve Sanderson's ES5 plugin so the brackets are not required in our markup bindings (albeit limited to IE9+)

---

I've included the code for our custom functions below. You'd need to include this code after you load Knockout, Breeze, and Steve's plugin above.

// Add a function to make an individual Breeze entity observable.
// Requires the Knockout ES5 plugin from http://blog.stevensanderson.com/2013/05/20/knockout-es5-a-plugin-to-simplify-your-syntax/
// Note: Breeze must be configured to use it's built in backingStore adapter: 
//     breeze.config.initializeAdapterInstance("modelLibrary", "backingStore", true);

(function () {

    var utils = breeze.utils = breeze.utils || {};

    utils.makeObservable = function (entity) {
        var bs = entity && entity._backingStore;
        if (!bs) {
            throw new Error("Only entities provided by the backingStore adapter can be made observable");
        }
        ko.track(bs);
    };

    utils.isObservable = function (entity) {
        var result = false,
            bs = entity && entity._backingStore;
        if (bs) {
            var propNames = Object.getOwnPropertyNames(bs);
            if (propNames.length > 0) {
                result = !!ko.getObservable(bs, propNames[0]);
            }
        }
        return result;
    };

})();