1
votes

I set up a MongoDb Sharded Replica Set with 2 clusters but I don't know how to test if everything works well.

I have no error messages on the any of the hosts, which suggest that my setup is ok. All hosts can ping each other and are using the default port 27017.

Here are the hostnames.

Shard 1:

  • mongo-rs1-1
  • mongo-rs1-2
  • mongo-rs1-3

Shard 2:

  • mongo-rs2-1
  • mongo-rs2-2
  • mongo-rs2-3

3 config servers:

  • cfg-1
  • cfg-2
  • cfg-3

A router (Mongos):

  • mongos-1

Here is what I have when running sh.status():

mongos> sh.status()  
                                                       
--- Sharding Status ---                                                         
  sharding version: {                                                           
        "_id" : 1,                                                              
        "version" : 4,                                                          
        "minCompatibleVersion" : 4,                                             
        "currentVersion" : 5,                                                   
        "clusterId" : ObjectId("562491c645300851d95d85bb")                      
}                                                                               
  shards:                                                                       
        {  "_id" : "rs1",  "host" : "rs1/mongo-rs1-1:27017,mongo-rs1-2:27017,mon
go-rs1-3:27017" }                                                               
        {  "_id" : "rs2",  "host" : "rs2/mongo-rs2-1:27017,mongo-rs2-2:27017,mon
go-rs2-3:27017" }                                                               
  databases:                                                                    
        {  "_id" : "admin",  "partitioned" : false,  "primary" : "config" }     
        {  "_id" : "testshard",  "partitioned" : true,  "primary" : "rs1" }                     
        {  "_id" : "testrs2",  "partitioned" : false,  "primary" : "rs2" }                              
                                                              

I run my mongos with commands:

mongos --configdb cfg-1:27017,cfg-2:27017,cfg-3:27017

I run my config servers with commands:

mongod --noprealloc --smallfiles --configsvr --dbpath /data/db --port 27017

I run my replicat sets (rs1 and rs2) with commands:

mongod --replSet rs1 --noprealloc --smallfiles

The replica sets are set like this:

rs1:PRIMARY> rs.status()                                                        
{                                                                               
        "set" : "rs1",                                                          
        "date" : ISODate("2015-10-19T15:45:31Z"),                               
        "myState" : 1,                                                          
        "members" : [                                                           
                {                                                               
                        "_id" : 0,                                              
                        "name" : "mongo-rs1-1:27017",                           
                        "health" : 1,                                           
                        "state" : 1,                                            
                        "stateStr" : "PRIMARY",                                 
                        "uptime" : 28658,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "electionTime" : Timestamp(1445240879, 1),              
                        "electionDate" : ISODate("2015-10-19T07:47:59Z"),       
                        "self" : true                                           
                },    
                {                                                               
                        "_id" : 1,                                              
                        "name" : "mongo-rs1-2:27017",                           
                        "health" : 1,                                           
                        "state" : 2,                                            
                        "stateStr" : "SECONDARY",                               
                        "uptime" : 28654,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "lastHeartbeat" : ISODate("2015-10-19T15:45:30Z"),      
                        "lastHeartbeatRecv" : ISODate("2015-10-19T15:45:30Z"),  
                        "pingMs" : 0,                                           
                        "syncingTo" : "mongo-rs1-1:27017"                       
                },  
                {                                                               
                        "_id" : 2,                                              
                        "name" : "mongo-rs1-3:27017",                           
                        "health" : 1,                                           
                        "state" : 2,                                            
                        "stateStr" : "SECONDARY",                               
                        "uptime" : 28654,                                       
                        "optime" : Timestamp(1445239630, 1),                    
                        "optimeDate" : ISODate("2015-10-19T07:27:10Z"),         
                        "lastHeartbeat" : ISODate("2015-10-19T15:45:29Z"),      
                        "lastHeartbeatRecv" : ISODate("2015-10-19T15:45:30Z"),  
                        "pingMs" : 1,                                           
                        "syncingTo" : "mongo-rs1-1:27017"                       
                }                                                               
        ],                                                                      
        "ok" : 1                                                                
}                          

Tests I made:

  1. I connected with my MongoDB client to the Mongos URL.
  2. I created a DB: "testshard"
  3. I created a document in "testshard"
  4. I logged in the 3 replica: mongo-rs1-* and mongo-rs2-* to check where the data was saved

Result

I noticed that the DB was created on RS1 (shard 1) and was replicated on the 3 replicas: mongo-rs1-1 , mongo-rs1-2, mongo-rs1-3. The DB was not created on the shard 2; I think this is normal as the sharding feature save on one shard but not both.

From here, it looks like the replica works. But what about the shard 2? How to check if data can be saved to shard 2?

Test #2

  1. I logged in the Mongos and I created the database rs2 which I force to rs2 in the database "config" of the Mongos.
  2. I create a document in testrs2.

I was expecting to see the new documents created in rs2, but rs2 is still an empty DB.

Questions

Q1: How can I test the "sharding" feature? I would expect to save a document and see it in the right replica set (rs1 or rs2).

Q2: How does Mongo know where to save the data between rs1 or rs2 ?

Q3: Is there a way to automatize the sharding? I don't want to tell which database should go where, this should be automatic. I was expecting the data to be saved randomly between the 2 shards.

Q4: What are the best backup strategies in my case? I guess I just need to do a mongodump of rs1 and rs2 ?

Q5: Am I right if I say that the only access point to the MongoDB is through the Mongos? Nobody should query the replica set.

1
Your question is off topic here it might be more suitable at dba.stackexchange.com but make sure to read their How to Ask page so you know what the site there is about.styvane

1 Answers

1
votes

Q1: How can I test the "sharding" feature? I would expect to save a document and see it in the right replica set (rs1 or rs2).

You have clipped sh.status() and omitted sharding information. See this post for what a shard status should look like. Know that the design includes regional shard tags, which if omitted would be similar to what you want. MongoDb regional replica set - primary node in each region?

Q2: How does Mongo know where to save the data between rs1 or rs2 ?

Your basic config identifies a shard key and mongo takes care of what documents go on what shard and rebalancing the shards as necessary.

Q3: Is there a way to automatize the sharding? I don't want to tell which database should go where, this should be automatic. I was expecting the data to be saved randomly between the 2 shards.

The sharding is "automatized" by default. When you shard a collection, you specify the shard key and mongo figures out how to partition the collection and re-balances as the database grows and document distribution between shard key ranges changes.

Q4: What are the best backup strategies in my case? I guess I just need to do a mongodump of rs1 and rs2 ?

Mongo dump on each shard member is a decent strategy for small databases, although using a decently distributed replica set eliminates the need for regular backups for everything except some types of maintenance (e.g. you are reshaping collections and you want a rollback database copy in case maintenance does not go well)

Q5: Am I right if I say that the only access point to the MongoDB is through the Mongos? Nobody should query the replica set.

All access must be to mongos. Only admins should access replica set, (or shard members) directly. Direct member access subverts sharding - results will vary with configuration.