db.cloneCollection ignore duplicate keys

Question

How do I clone a collection with MongoDB and ignore duplicate keys?

$ mongo items
MongoDB shell version: 2.4.6
connecting to: items
> db.cloneCollection('localhost:27018', 'things')
{
    "errmsg" : "exception: E11000 duplicate key error index: items.things.$_id_  dup key: { : ObjectId('52558bebdedc25038ed26d58') }",
    "code" : 11000,
    "ok" : 0
}

Better yet, is there a safer way of merging a remote collection with a local one? If db.cloneCollection is interrupted, there doesn't seem to be a way to "resume" it without wiping out all of the duplicate items and restarting it from the beginning.

Alan Bogu Alan Bogu · Accepted Answer · 2015-11-24T11:27:52

You can create another collection named say "things2" and clone there the remote collection. Then use unordered bulk insert to the "things" collection for each document of "things2" collection - it will ignore duplicate key errors until the whole bulk insert is done.

db.cloneCollection('localhost:27018', 'things2');

var cursor = db.things2.find(); null;

var bulk = db.things.initializeUnorderedBulkOp();


cursor.forEach(function(doc) {
  bulk.insert(doc);
});

bulk.execute();

or you can create an array with all the documents from "things2" collection and then "insert" it to the "things" collection with the option { ordered: false }

db.cloneCollection('localhost:27018', 'things_2');

var things2array = db.things2.find().toArray(); null;

db.things.insert(things2array,{ ordered : false });

db.cloneCollection ignore duplicate keys

1 Answers