How to make Backbone.js Collection items Unique?

Question

Say I have these Backbone.js Model:

var Truck = Backbone.Model.extend({});

var truck1 = new Truck();
var truck2 = new Truck();

truck1.set("brand", "Ford");
truck2.set("brand", "Toyota");
truck3.set("brand", "Honda");
truck4.set("brand", "Ford");

Then, let's say we have a Backbone.js Collection:

var TruckList = Backbone.Collection.extend({
  model: Truck,
  comparator: function(truck) {
     return truck.get("brand");
  };

});

I'm a car collector, so time to add each car to my collection:

Trucks = new TruckList();
Trucks.add(truck1);
Trucks.add(truck2);
Trucks.add(truck3);
Trucks.add(truck4);

Just focusing on the brand attribute, truck4 is a duplicate of truck1. I can't have duplicates in my Collection. I need my collection to have unique values.

My question is, How do I remove duplicate items from my Backbone.js Collection?

Should I use Underscore.js for this? If so, can someone please provide a working/runnable example of how to do this.

Assume the following:

1.Collection is not sorted

1. Removal must be done on brand attribute value

2. Ajax call to populate each instance of a Truck. This means when adding to a collection, you don't have access to the Truck properties.

Answer 1

I would override the add method in your TruckList collection and use underscore to detect duplicates there and reject the duplicate. Something like.

TruckList.prototype.add = function(truck) {
    // Using isDupe routine from @Bill Eisenhauer's answer
    var isDupe = this.any(function(_truck) { 
        return _truck.get('brand') === truck.get('brand');
    });

    // Up to you either return false or throw an exception or silently ignore
    // NOTE: DEFAULT functionality of adding duplicate to collection is to IGNORE and RETURN. Returning false here is unexpected. ALSO, this doesn't support the merge: true flag.
    // Return result of prototype.add to ensure default functionality of .add is maintained. 
    return isDupe ? false : Backbone.Collection.prototype.add.call(this, truck);
}

Answer 2

The simplest way to achieve this is to make sure the models you are adding have unique ids. By default Backbone collections will not add models with duplicate ids.

test('Collection should not add duplicate models', 1, function() {
    var model1 = {
        id: "1234"
    };
    var model2 = {
        id: "1234"
    };

    this.collection.add([model1, model2]);

    equal(1, this.collection.length, "collection length should be one when trying to add two duplicate models");
});

Answer 3

Try this. It uses the any underscore method to detect the potential duplicate and then dumps out if so. Of course, you might want to dress this up with an exception to be more robust:

TruckList.prototype.add = function(newTruck) {
  var isDupe = this.any(function(truck) { 
    return truck.get('brand') === newTruck.get('brand');
  }
  if (isDupe) return;
  Backbone.Collection.prototype.add.call(this, truck);
}

As an aside, I would probably write a function on Truck to do the dupe checking so that the collection doesn't know too much about this condition.

Answer 4

var TruckList = Backbone.Collection.extend({
model : Truck,

// Using @Peter Lyons' answer
add : function(truck) {
    // Using isDupe routine from @Bill Eisenhauer's answer
    var isDupe = this.any(function(_truck) {
        return _truck.get('brand') === truck.get('brand');
    });
    if (isDupe) {
        // Up to you either return false or throw an exception or silently
        // ignore
        return false;
    }
    Backbone.Collection.prototype.add.call(this, truck);
},

comparator : function(truck) {
    return truck.get("brand");
} });

VassilisB's answer worked great but it will override Backbone Collection's add() behavior. Therefore, errors might come when you try to do this:

var truckList = new TruckList([{brand: 'Ford'}, {brand: 'Toyota'}]); 

So, I added a bit of a checking to avoid these errors:

var TruckList = Backbone.Collection.extend({
    model : Truck,

    // Using @Peter Lyons' answer
    add : function(trucks) {
        // For array
        trucks = _.isArray(trucks) ? trucks.slice() : [trucks]; //From backbone code itself
        for (i = 0, length = trucks.length; i < length; i++) {
            var truck = ((trucks[i] instanceof this.model) ? trucks[i]  : new this.model(trucks[i] )); // Create a model if it's a JS object

            // Using isDupe routine from @Bill Eisenhauer's answer
            var isDupe = this.any(function(_truck) {
                return _truck.get('brand') === truck.get('brand');
            });
            if (isDupe) {
                // Up to you either return false or throw an exception or silently
                // ignore
                return false;
            }
            Backbone.Collection.prototype.add.call(this, truck);
       }
    },

    comparator : function(truck) {
        return truck.get("brand");
    }});

Answer 5

I'm doing a FileUpload thing with the same issue, and here's how I did it (coffeescript):

File = Backbone.Model.extend
    validate: (args) ->
        result
        if [email protected](args)
            result = 'File already in list'
        result

Files = Backbone.Collection.extend
    model: File

    isUniqueFile: (file) ->
        found
        for f in @models
            if f.get('name') is file.name
                found = f
                break
        if found
            false
        else
            true

... and that's it. The collection object is automatically referenced in File, and Validation is automatically called when an action is invoked on the collection which in this case is Add.

Answer 6

Underscore.js, a pre-req for backbone.js, provides a function for this: http://documentcloud.github.com/underscore/#uniq

Example:

_.uniq([1,1,1,1,1,2,3,4,5]); // returns [1,2,3,4,5]

Answer 7

Not sure if this is an update to either Backbone or underscore, but the where() function works in Backbone 0.9.2 to do the matching for you:

TruckList.prototype.add = function(truck) {
    var matches = this.where({name: truck.get('brand')});

    if (matches.length > 0) {
        //Up to you either return false or throw an exception or silently ignore
        return false;
    }

    Backbone.Collection.prototype.add.call(this, truck);
}

Answer 8

I would prefer override the add method like this.

var TruckList = Backbone.Collection.extend({
model : Truck,

// Using @Peter Lyons' answer
add : function(truck) {
    // Using isDupe routine from @Bill Eisenhauer's answer
    var isDupe = this.any(function(_truck) {
        return _truck.get('brand') === truck.get('brand');
    });
    if (isDupe) {
        // Up to you either return false or throw an exception or silently
        // ignore
        return false;
    }
    Backbone.Collection.prototype.add.call(this, truck);
},

comparator : function(truck) {
    return truck.get("brand");
} });

Answer 9

It seems like an elegant solution would be to use _.findWhere so long as you have some unique attribute (brand in your case). _.findWhere will return a match which is a JavaScript object and therefore truthy or undefined which is falsey. This way you can use a single if statement.

var TruckList = Backbone.Collection.extend({
  model: Truck,

  add: function (truck) {
    if (!this.findWhere({ brand: truck.get('brand') })) {
      Backbone.Collection.prototype.add.call(this, truck);
    }
  }
});

Answer 10

Try this...

var TruckList = Backbone.Collection.extend({
  model: Truck,
  comparator: function(truck) {
     return truck.get("brand");
  },
  wherePartialUnique: function(attrs) {
    // this method is really only tolerant of string values.  you can't do partial
    // matches on arrays, objects, etc.  use collection.where for that
      if (_.isEmpty(attrs)) return [];
      var seen = [];
      return this.filter(function(model) {
        for (var key in attrs) {
          // sometimes keys are empty. that's bad, so let's not include it in a unique result set
          // you might want empty keys though, so comment the next line out if you do.
          if ( _.isEmpty(model.get(key).trim()) ) return false;
          // on to the filtering...
          if (model.get(key).toLowerCase().indexOf(attrs[key].toLowerCase()) >= 0) {
            if (seen.indexOf( model.get(key) ) >= 0 ) return false;
            seen.push(model.get(key));
            return true;
          } else {
            return false;
          }
        }
        return true;
      });
    }
});

A few things to remember:

1. this is based on the backbone.collection.where method and unlike that method, it will attempt partial matches on model attributes within a collection. If you don't want that, you'll need to modify it to only match exactly. Just mimic what you see in the original method.

2. it should be able to accept multiple attribute matches, so if you have model attributes of foo and bar, you should be able to do collection.wherePartialUnique({foo:"you",bar:"dude"}). I have not tested that though. :) I have only ever done one key/value pair.

3. i also strip out empty model attributes from consideration. I don't care about them, but you might.

4. this method doesn't require a collection of unique model properties that the comparator depends. It's more like a sql distinct query, but I'm not an sql guy so don't shoot me if that's a bad example :)

5. your collection is sorted by way of the comparator function, so one of your assumptions about it not being sorted is incorrect.

I believe this also addresses all of your goals:

1. Collection is not sorted
2. Removal must be done on brand attribute value
3. Ajax call to populate each instance of a Truck. This means when adding to a collection, you don't have access to the Truck properties.

Answer 11

I'm really unhappy with the accepted answer to this solution. It contains numerous errors. I've edited the original solution to highlight my concerns, but I am proposing the following solution assuming you're OK dirtying your duplicate's id/cid property:

TruckList.prototype.add = function(truckToAdd, options) {
    //  Find duplicate truck by brand:
    var duplicateTruck = this.find(function(truck){
        return truck.get('brand') === truckToAdd.get('brand');
    });

    //  Make truck an actual duplicate by ID: 
    //  TODO: This modifies truckToAdd's ID. This could be expanded to preserve the ID while also taking into consideration any merge: true options.
    if(duplicateTruck !== undefined){
        if(duplicateTruck.has('id')){
            truckToAdd.set('id', duplicateTruck.get('id'), { silent: true });
        }
        else {
            truckToAdd.cid = duplicateTruck.cid;
        }
    }

    //  Allow Backbone to handle the duplicate instead of trying to do it manually.
    return Backbone.Collection.prototype.add.call(this, truckToAdd, options);
}

The only flaw with this one is that truckToAdd's ID/cid is not preserved. However, this does preserve all of the expected functionality of adding an item to a collection including passing merge: true.

Answer 12

I was not satisfied with the provided answers for several reasons:

• Modifying the return value of add is unexpected.
• Not supporting { merge: true } is unexpected.

I've provided a solution which I believe to be more robust. This solution clones given models if they have duplicates in the collection, updates the clones' ID to match the duplicates ID, and then passes the list of duplicates and non-duplicates onto the original add method so that it can do its magic. No unintended side-effects as far as I am aware.

add: function (models, options) {
    var preparedModels;

    if (models instanceof Backbone.Collection) {
        preparedModels = models.map(this._prepareModelToAdd.bind(this));
    }
    else if (_.isArray(models)) {
        preparedModels = _.map(models, this._prepareModelToAdd.bind(this));
    } else if (!_.isNull(models) && !_.isUndefined(models)) {
        preparedModels = this._prepareModelToAdd(models);
    } else {
        preparedModels = models;
    }

    //  Call the original add method using preparedModels which have updated their IDs to match any existing models.
    return Backbone.Collection.prototype.add.call(this, preparedModels, options);
},

//  Return a copy of the given model's attributes with the id or cid updated to match any pre-existing model.
//  If no existing model is found then this function is a no-op.
//  NOTE: _prepareModel is reserved by Backbone and should be avoided.
_prepareModelToAdd: function (model) {
    //  If an existing model was not found then just use the given reference.
    var preparedModel = model;
    var existingModel = this._getExistingModel(model);

    //  If an existing model was found then clone the given reference and update its id.
    if (!_.isUndefined(existingModel)) {
        preparedModel = this._clone(model);
        this._copyId(preparedModel, existingModel);
    }

    return preparedModel;
},

//  Try to find an existing model in the collection based on the given model's brand.
_getExistingModel: function (model) {
    var brand = model instanceof Backbone.Model ? model.get('brand') : model.brand;
    var existingModel = this._getByBrand(brand);
    return existingModel;
},

_getByBrand: function (brand) {
    return this.find(function (model) {
        return model.get('brand') === brand;
    });
},

_clone: function (model) {
    //  Avoid calling model.clone because re-initializing the model could cause side-effects.
    //  Avoid calling model.toJSON because the method may have been overidden.
    return model instanceof Backbone.Model ? _.clone(model.attributes) : _.clone(model);
},

//  Copy the model's id or cid onto attributes to ensure Backbone.Collection.prototype.add treats attributes as a duplicate.
_copyId: function (attributes, model) {
    if (model.has('id')) {
        attributes.id = model.get('id');
    } else {
        attributes.cid = model.cid;
    }
}