MongoDb GridFS with the existing specific collection in a database

Question

Consider the below "restaurants" collection documents in the "restaurant" mongodb database.

{
"_id" : ObjectId("55f6564073fc0a09338cf434"),
"address" : {
    "building" : "97-22",
    "coord" : [ 
        -73.8601152, 
        40.7311739
    ],
    "street" : "63 Road",
    "zipcode" : "11374"
},
"borough" : "Queens",
"cuisine" : "Jewish/Kosher",
"grades" : [ 
    {
        "date" : ISODate("2014-11-23T17:30:00.000-06:30"),
        "grade" : "Z",
        "score" : 20
    }, 
    {
        "date" : ISODate("2013-01-16T17:30:00.000-06:30"),
        "grade" : "A",
        "score" : 13
    }, 
    {
        "date" : ISODate("2012-08-01T17:30:00.000-06:30"),
        "grade" : "A",
        "score" : 13
    }, 
    {
        "date" : ISODate("2011-12-14T17:30:00.000-06:30"),
        "grade" : "B",
        "score" : 25
    },

   ........
   .......
   .......
     {
        "date" : ISODate("2011-12-14T17:30:00.000-06:30"),
        "grade" : "AZZ",
        "score" : 25
    },   -- It reaches 15 MB
],
"name" : "Tov Kosher Kitchen",
"restaurant_id" : "40356068"

}

In this document "grades" filed is the embedded document array. This array is going to reach mongodb maximum document size 16Mb. So now we are in the situation to alter the data model for this collection. I came to know that in the mongoDb we can store the documents which exceed the default limit of 16mb by using "gridfs" . I'm not able to find the links to demonstrate this. I need to store gridfs documents along with existing fields in the collection.

You certainly don't want GridFS, as it simply implements "binary" storage of data in chunks ( under 16MB ) and an interface to store and retrieve that data as whole content. There is no facility to "query" the inner content as you can with document fields. If your array data will really be "too big" then you are better off storing in another collection. But the questions you really should be asking yourself are 1. Is it "really" going to be too big? As it takes a lot of data to reach 16MB. 2. Do I really need to store all of this anyway? Or is keeping aggregated totals going to be enough? — Blakes Seven
You seem to be attemptlng a long reply. Please rather add details to your question instead and just alert commenters of changes. You are basically misunderstanding what GridFS is, so I suggest reading some documentation. You either want a separate collection or to reconsider what is necesary to store or it's eventual realistic growth. — Blakes Seven
Hi Blakes Seven, Thank you for the update. to answering your questions 1. Is it "really" going to be too big? As it takes a lot of data to reach 16MB : "Yes" . 2.Do I really need to store all of this anyway? Or is keeping aggregated totals going to be enough? : "All need to be store in this way" .So GridFs cannot be used in this case then what could be the alternative idea? A new colletion with Joins to the existing collection is the only alternative ? or there is any other way to go.. Please advice — Root
I stilll don't think you are being realistic. The data you propose would need 10's of thousands of entries to even approach 16MB ( not that you really should be doing that anyway ). The big guide I want to give is look through the questions and answers on this very site. Not one entry has combined answer or comment detail that would breach 16MB of storage if put into a MongoDB document. You would not be the first to propose your new "killer app" is going to get more traffic or usage than StackOverflow or others. But I strongly doubt it. — Blakes Seven

Markus W Mahlberg Markus W Mahlberg · Accepted Answer · 2015-09-28T06:17:18

Just because you can embed documents, it doesn't mean that it's always a good idea. More often than not, it isn't.

Markus W Mahlberg

It basically only works in a "One-to-(Very-)Few" relationship. In your case, it's different.

Ask the right question. What do you really want to know? My guess is

What are the grades for a given restaurant?

So, modeling becomes easy. First, a restaurants collection

{
  _id: someObjectId,
  name: "The Foo Bar",
  cuisine: "Foo and Baz",
  ...
}

and a grades collection:

{
  _id: new ObjectId(),
  restaurant: someObjectId,
  grade: "A",
  date: someISODate
}

Answering the question is nothing more than:

db.grades.find(
  { restaurant: givenRestaurantsObjectId }
)

MongoDb GridFS with the existing specific collection in a database

1 Answers