MongoDB mapReduce: Emit a DBRef oid as key

Question

I need to maintain an application where data stored in MongoDB, and I'm fluent yet with the mapReduce syntax.

I want to make a mapReduce operation where the emitted key is the ID of a DBRef, not it's value. I clearly understand from MongoDB documentation that for parallelism reason one mustn't and can't access the DB.

So my collection documents are like that:

{
  _id: "66f072b8-2422-4022-826b-b20b832a1ee6",
  _class: "com.foo.bar",
  foo_bar: 1,
  user: {
    "$dbref": {
      namespace: "user",
      oid: "6cd0dac5-a511-48b1-b437-318ad74061a5"
    }
  }
}

The current mapReduce is like that:

db.myCollection.mapReduce(
  function () {
      if (this.foo_bar > 0) {
          emit(this.user, this.foo_bar);
      }
  },
  function (key, values) {
      var sum = 0;
      for (var i = 0; i < values.length; i++) {
          sum += values[i];
      }
      return sum;
  },

  {
    "out" : { "inline" : 1} 
  }
)

There is a lot of documents and the loading between AWS and MongoHQ is taking a lot of time. I would like to be able to use user$id as a key, instead of the DBRef's document.

I've tested with the following but without success:

emit(this.user.id, this.foo_bar);
emit(this.user.oid, this.foo_bar);
emit(this.user._id, this.foo_bar);
emit(this.user.dbref.id,this.foo_bar);
emit(this.user.dbref.oid, this.foo_bar);
emit(this.user.dbref._id, this.foo_bar);
emit(this.user['$dbref'].oid, this.foo_bar); (as per Sammaye suggestion in comments)

What's the correct syntax?

Nope. Mongo gives: Thu Jan 30 23:22:52.584 map reduce failed:{ "errmsg" : "exception: TypeError: Cannot read property 'oid' of undefined near '['$dbref'].oid, this. foo_bar)' (line 3)", "code" : 16722, "ok" : 0 } at src/mongo/shell/collection.js:970 - any suggestion @cubitouch? — Gammatora
I think the biggest problem here is that the Java driver have not standardised their dbrefs, I assume you use java from the other details in the document — Sammaye
We indeed use Java (Spring MongoDB). The output I'm giving you is by using mongo cli on OSX. — Gammatora
Hang on Ima test this, there is no explanation why this shouldn't work — Sammaye

Neil Lunn Neil Lunn · Accepted Answer · 2014-01-30T23:01:59

The real answer here I believe is revisit your schema design. It is noted that the Spring data approach is more skewed to a traditional relational design rather than the dynamic document structure MongoDB provides.

That said, here is a link with more explanation and alternatives from the MongoDB community manager. Follow the notes and the link contained in that response.

The only approach other than re-design, and this is really only a workaround to actually doing the re-design is to pre-join the data using the output from map reduce. This is an incremental step. The resulting collection will allow you to query the properties of the sub-documents without pulling each document down over the wire.

As another note a similar output option is set to be added to the Aggregation Framework pipeline in the next release of MongoDB.

MongoDB mapReduce: Emit a DBRef oid as key

1 Answers