Should I split NoSQL database on separate servers?

Question

We are still in the early stages of determining whether we go with RDBMS or NoSQL.

One of the areas of interest is if we go with NoSQL (probably CouchDB although could be MongoDB), would separate NoSQL database on different servers be better than one instance of a NoSQL server?

We will be building a file management system where certain files/videos will be grouped on different servers. Accounts related files/videos will be stored on the accounts server etc. To query for an accounts related file we will most likely search the database on the accounts server.

I can see in the future that someone will say "why cant I search over all servers for a type of file or video"?

Clearly having one database would be better here. However apart from the latency in http requests to query the servers, are there better ways to do this or pros and cons of having a large database?

JD

Answer 1

The idea of (most) NoSQL products is that they provide horizontal scalability. This means that you single logical instance could be on dozens of servers. In MongoDB, for example, you can use auto-sharding. For your program, this is completely transparent: Your code is (almost) the same you'd use for a single database server, but the data resides on, say, 5 servers.

There are many upsides: you have a central spot to administer your database, you can query across all databases if necessary, you don't need to mess around w/ multiple db connections in your code, the db automatically balances those collections that need to be balanced, map/reduce operations can run in parallel if the query allows it, etc.

The most important ones for me: there's not much administration overhead, and you don't have to put too much thought into this now, because you can add auto-sharding later.

I would not try to do sharding myself, because it is re-inventing the wheel, and it's not easy either. This was one of the key drivers for NoSQL in the first place.