Ember.js/Ember Data takes several minutes to update hasMany of parent after creating thousands of child records

Question

I'm new to Ember.js and running into performance issues when trying to create thousands of records (5300 to be exact) and updating a hasMany relationship. I'm making a request to my API to retrieve records before I create my new records. After the promise returns, I then do a forEach over each record brought down (5300) to do my calculations for the new set of records. Creating the records themselves takes about 2 seconds. Updating the hasMany starts off quickly only for the first 40 or so records and then slows to about one update per second.

I should also note that this is being done within a component. I know this is typically anit-pattern, but in this case there is no reason to change the URL or transition. This is a screen where users can select from a pool of items (provided by the route), apply a pricing rule, and then create an event (created in the route) based on those items. It's a sandbox to determine what items will be a part of the event. Once the users have decided on their items for the event, I then send an action up to the route to perform the actual save and persist to my backend. Outside of the anti-pattern aspect, I can't see how this would affect the performance of the hasMany update.

I'm using RESTAdapter and RESTSerializer for what it's worth, but that should not have any impact here since I'm just dealing with the Ember data store.

Ember Version:

Ember             : 2.5.1
Ember Data        : 2.5.3
jQuery            : 2.2.3
Ember Simple Auth : 1.1.0

The two models in question are as follows...

Child Model (event-item):

export default DS.Model.extend({
  event: DS.belongsTo('event'),
  itemNumber: DS.attr('string'),
  styleNumber: DS.attr('string'),
  tier: DS.attr('string'),
  eventPrice: DS.attr('number')
});

Parent Model (event):

export default DS.Model.extend({
  eventTypeId: DS.attr('string'),
  eventName: DS.attr('string'),
  eventDesc: DS.attr('string'),
  startDate: DS.attr('moment-date'),
  endDate: DS.attr('moment-date'),
  priority: DS.attr('number'),
  statusCode: DS.attr('string'),
  value: DS.attr('number'),
  eventItems: DS.hasMany('event-item', {async:true})
});

Event create record:

model() {
    return this.store.createRecord('event', {});
},

Code block in component responsible for creating records and updating hasMany:

this.get('store').query('pricing', {brandCd: '00'}).then(tiers => {
    tiers.forEach(tier => {
        this.get('event').get('eventItems').createRecord({
            styleNumber: tier.get('styleNumber'),
            itemNumber: tier.get('itemNumber'),
            brandCd: '00',
            tier: tier.get('tier'),
            eventPrice: this._calculateEventPrice(tier.get('origPrice'), this.get('event').get('eventTypeId'), this.get('event').get('value')),
        });
    });

    this.get('event').set('needsUpdated', 'Y');
});

So far I've tried the following...

Adding inverse relationships to my hasMany and belongsTo
Adding all of the create records to an Ember.A() and then trying to push the new records to the hasMany like so: this.get('event').get('eventItems').pushObjects(newEventItems);. Also tried it using this.get('event').get('eventItems').addObjects(newEventItems);.
Setting the belongsTo on the record being created instead of updating the hasMany of the parent (event).
I also went ahead and moved this logic into my route just to make sure I wasn't getting odd behavior by doing this in the component. It performs the same.

I would assume (and please correct me if I'm wrong) that creating records and updating relationships strictly on the client side should be able to handle thousands of records without too much issue. I'm hoping I'm just doing something incorrect or in an inefficient way that will be obvious to someone with more experience. Any help, including alternatives, is greatly appreciated!

Tim Tim · Accepted Answer · 2016-05-29T14:33:47

I've found working with large sets of 'hasMany' relationships to be slow as well.

My advice: Build the relationship on the server with a custom endpoint, and communicate the changes back to the client over websockets. Trying to individually save 5300 records in Ember will make 5300 network requests, whereas this could be accomplished in 1 outbound request, and possibly one websockets message, though it might be advisable to batch the responses in smaller sets.

No need to make the relationships in Ember until the changes are committed. The returned websockets message should include the relationship's foreign keys Ember Data needs to construct the relationships.

This means that up front, you'll be creating eventItems (or whatever your model is called) without a parent.

Finally, you might consider creating the child records on demand, instead of trying to create every child item for the user right away. Just do the createRecord when a user decides to select the item from the pool of items. (Hopefully I'm understanding your use case correctly.) If you take that approach, you might not even need to use a custom endpoint as I've described.

Last bit of advice: Your note about 'anti-patterns' is correct: NEVER do CRUD operations, or anything really async, in components, especially not for 5300 items. Components can easily be torn down before an async operation completes, leaving your application in a weird state, and most likely resulting in bugs. Move whatever you do into the route, like you said, and stick with that pattern. Components should just be "dumb" templates that display stuff and send actions.

Ember.js/Ember Data takes several minutes to update hasMany of parent after creating thousands of child records

1 Answers