MongoDB sharding not distributing optimally

Question

I am working on a test MongoDB implementation where I am trying to bulk insert 1,000,000 records into a collection and have it distributed evenly between two shards. My initial trials saw one shard containing 995760 records and the other shard only containing 4251 records. I attempted to pre-split but that did not change anything. I'm new to the concept of sharding and would appreciate any help on the subject.

UPDATE:

My shard key in on the field "number" which is an integer that I have ranging from 1 - 999,999

Status:

{
"sharded" : true,
"ns" : "test.test_collection",
"count" : 999999,
"numExtents" : 21,
"size" : 43982976,
"storageSize" : 210247680,
"totalIndexSize" : 60396112,
"indexSizes" : {
    "_id_" : 32466896,
    "number_1" : 27929216
},
"avgObjSize" : 43.983019983019986,
"nindexes" : 2,
"nchunks" : 239,
"shards" : {
    "firstset" : {
        "ns" : "test.test_collection",
        "count" : 995754,
        "size" : 43813176,
        "avgObjSize" : 44,
        "storageSize" : 123936768,
        "numExtents" : 11,
        "nindexes" : 2,
        "lastExtentSize" : 37625856,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 0,
        "totalIndexSize" : 60118128,
        "indexSizes" : {
            "_id_" : 32319728,
            "number_1" : 27798400
        },
        "ok" : 1
    },
    "secondset" : {
        "ns" : "test.test_collection",
        "count" : 4245,
        "size" : 169800,
        "avgObjSize" : 40,
        "storageSize" : 86310912,
        "numExtents" : 10,
        "nindexes" : 2,
        "lastExtentSize" : 27869184,
        "paddingFactor" : 1,
        "systemFlags" : 1,
        "userFlags" : 0,
        "totalIndexSize" : 277984,
        "indexSizes" : {
            "_id_" : 147168,
            "number_1" : 130816
        },
        "ok" : 1
    }
},
"ok" : 1
}

UPDATE 2:

Thanks to @Sammaye for the suggestion. The issue had to do with my shard key. When I hashed the key, the bulk insert split the records evenly. Thanks for all the help!

We are gonna need a lot of info here, can you start by editing with a status()? Also tell us what your shard key is — Sammaye
One thing I noticwed is that your shard key is montonic, which means it goes from 1-999,999, this could mean that sharding is working the way it is meant to and shard 1 has become full and as such has spilled over to shard 2. If you tried hashed keys does it change the distribution of your data? — Sammaye
When you say hash the shard key, I assume you mean hashing on a key that has been hashed in order to get an even spread? Or do you mean that there is a flag on the sharding setup which lets you activate hashing? — Rich
There is a flag you can set to actually hash a key now: blog.mongodb.org/post/47633823714/… , glad to have helped :) — Sammaye

Derick Derick · Accepted Answer · 2013-07-25T09:42:43

It generally takes some time for the cluster to become balanced. If you have input all those documents in one big insert then they will first be all written to the primary shard and it will be split up in chunks once it hits the max chunk size. Chunks are then migrated to the other shard one by one and that might takes some time.

From what I can see you have about 100MB of data, but 429 chunks, which is a bit strange? What have you set as chunk size? A too small chunk size can definitely slow down the migration of chunks to the other shard.

MongoDB sharding not distributing optimally

1 Answers