mongoimport E11000 duplicate key due to index

Question

I have a collection with the following fields:

name
type
color

I also have a unique index name_1_type_1.

Assuming a data set:

[{
  name: "name1",
  type: "type1",
  color: "blue"
}, {
  name: "name2",
  type: "type1",
  color: "green"
}]

Using mongoimport I create the initial data set.

Now, I need to update the collection in order to achieve the following 3 goals:

insert new documents (e.g. name1-type2 in snippet below)
update color in existing documents (e.g. blue->red in name1 below)

append a new optional field shape in some documents

[
    {
        name: "name1",
        type: "type1",
        color: "red",
        shape: "circle"
    }, 
    {
        name: "name1",
        type: "type2",
        color: "green",
        shape: "rectangle"
    }
]

However, when executing mongoimport --upsert on the above json file, I get:

error inserting documents: E11000 duplicate key error collection: test.col1 index: name_1_type_1 dup key

Maybe I am using mongoimport in a wrong way.

How can I achieve the 3 above mentioned upsert goals using mongoimport?

Blakes Seven Blakes Seven · Accepted Answer · 2016-03-26T07:06:44

You seem to have missed the --upsertFields option. Without it mongoimport assumes you mean _id and especially if that is not present in the file being imported, then it's just trying to "insert" new items all the time. Hence the duplicate key error.

So if you specify the fields on which you have based the unique key:

mongoimport -d database -c collection --upsert --upsertFields name,type input.json

Then you should get a result like the following:

{
        "_id" : ObjectId("56f6332a49ec4ea8330063b6"),
        "name" : "name1",
        "type" : "type1",
        "color" : "red",
        "shape" : "circle"
}
{
        "_id" : ObjectId("56f6332a49ec4ea8330063b7"),
        "name" : "name2",
        "type" : "type1",
        "color" : "green"
}
{
        "_id" : ObjectId("56f633d4824b97f80d3714b1"),
        "name" : "name1",
        "type" : "type2",
        "color" : "green",
        "shape" : "rectangle"
}

Note that in modern releases the --upsert is implied when you use --upsertFields.

N.B. You might also need --jsonArray if your data is structured that way.

mongoimport E11000 duplicate key due to index

1 Answers