Merge RDBMS and NoSQL database

4
votes

Ok, let's say I have a large MySql database with 100+ tables. Although it runs very well I have two tables that can each exceed 100 million entries per year. On these two tables I have to perform searches and calculations, not just archive information.

I was thinking of tranfering the logic of these tables to a NoSQL database due to it's good scalability.(MongoDB, CouchDB)

Since the information stored in these tables is used in many queries through the application I was thinking if there is a way to create a merge between the two databases.

Although it would be too beautiful to be able to write a sql query that automatically pulls data from the nosql db maybe there is another approach; like using a stored plsql procedure in MySQL to call a query on the NoSQL database. Or calling a system command from plsql and getting the results.

Just want to mention, I used MySQL as an example, it can be anything, Oracle, Postgres.

1
mongodbrelational-databasenosql
Have you heard of Sqoop (sqoop.apache.org) to transfert data between RDBMS and Hadoop ? - Kevin
No, but I will deffinetly look into it. - ionutab
I'm not getting you. You want to transfer half data of these too tables from RDMBS to nosql or full. If full, then why you want to redirect query. - Ravi Khakhkhar
No, not full, half from nosql, half from rdbms. - ionutab
On a side note, there is a Dissertation Project proposal in my university on merging RDMS and NoSQL databases. seems prety interesting. - ali

1 Answers

1
votes

So the buzzword for doing something like this is Polyglot Persistence (maybe this helps for further research). Mostly you will have to do the integration of this different datastores by yourself. There are two solutions I am aware of, I haven't tested them though.

The first one is using EclipseLink as an abstraction above the two datastores. The first link i found for that: http://blog.eisele.net/2012/11/polyglot-persistence-eclipselink-with.html

The second one is using PostgreSQL and Foreign data wrappers to kind of nest a nosql database within Postgres: http://wiki.postgresql.org/wiki/Foreign_data_wrappers