3
votes

I have 1 million documents in mongodb. I want to find and unset same fields. Can you give me a way or an idea?

My documents like this:

{
        "regions" : [ 
            {"id" : "1", "name" : "World"}, 
            {"id" : "10370","name" : "South America"}, 
            {"id" : "1426","name" : "Suriname"}
        ]
    }
    {
        "regions" : [ 
            {"id" : "1", "name" : "World"}, 
            {"id" : "10370","name" : "South America"}, 
            {"id" : "1426","name" : "Suriname"}
        ]
    }
    {
        "regions" : [ 
            {"id" : "1","name" : "World"}, 
            {"id" : "1734","name" : "USA"}, 
            {"id" : "1136","name" : "Pennsylvania"}, 
            {"id" : "16962","name" : "Greater Philadelphia area"}, 
        ]
    }
    {
        "regions" : [ 
            {"id" : "1","name" : "World"}, 
            {"id" : "1734","name" : "USA"}, 
            {"id" : "1136","name" : "Pennsylvania"}, 
            {"id" : "16962","name" : "Greater Philadelphia area"}, 
        ]
    }
    {
    "regions" : [ 
        {"id" : "1","name" : "World"}, 
        {"id" : "34964","name" : "Oceania"}, 
        {"id" : "15","name" : "Australia"}, 
        {"id" : "470","name" : "Western Australia"}, 
        {"id" : "36282","name" : "Perth"}, 
      ]
   }

How to I change like this:

{
        "regions" : [ 
            {"id" : "1", "name" : "World"}, 
            {"id" : "10370","name" : "South America"}, 
            {"id" : "1426","name" : "Suriname"}
        ]
    }
    {
        "regions" : [ 
            {"id" : "1","name" : "World"}, 
            {"id" : "1734","name" : "USA"}, 
            {"id" : "1136","name" : "Pennsylvania"}, 
            {"id" : "16962","name" : "Greater Philadelphia area"}, 
        ]
    }
    {
"regions" : [ 
    {"id" : "1","name" : "World"}, 
    {"id" : "34964","name" : "Oceania"}, 
    {"id" : "15","name" : "Australia"}, 
    {"id" : "470","name" : "Western Australia"}, 
    {"id" : "36282","name" : "Perth"}, 
   ]
  }

Thanks for your answer and interest in advance.

UPDATE: I'm trying this code:

db.collection.aggregate(
 {"$group":{"_id": {"id": "$regions.id","name": "$regions.name"},}},
 {"$group":{"_id":ObjectId(),"regions": { "$push": {"id": "$_id.id","name": $_id.name"}}}},
 {"$unwind": "$regions"},
 {"$out": "newcollection"}
)

It gives this error: "errmsg" : "insert for $out failed: { connectionId: 111, err: \"E11000 duplicate key error collection: collection.tmp.agg_out.12 index: id dup key: { : ObjectId('5767f378ff8f5e9302d95bc8') }\", code: 11000, n: 0, ok: 1.0 }",

How can I give a unique key?

1
It's unclear what you want to do here. Do you want to delete the duplicates from the collection?styvane
Yes I want delete duplicate field.tabarly

1 Answers

0
votes

Using aggregation, If you group by array elements, you can get rid of the duplicate region. something like this should help?

db.regs.aggregate([{$group:{"_id":{id:"$regions.id",name:"$regions.name"}}}]).pretty()