MongoDB MapReduce producing different results for each document

Question

This is a follow-up from this question, where I tried to solve this problem with the aggregation framework. Unfortunately, I have to wait before being able to update this particular mongodb installation to a version that includes the aggregation framework, so have had to use MapReduce for this fairly simple pivot operation.

I have input data in the format below, with multiple daily dumps:

"_id" : "daily_dump_2013-05-23",
    "authors_who_sold_books" : [
        {
            "id" : "Charles Dickens",
            "original_stock" : 253,
            "customers" : [
                {
                   "time_bought" : 1368627290,
                   "customer_id" : 9715923
                }
            ]
        },
        {
            "id" : "JRR Tolkien",
            "original_stock" : 24,
            "customers" : [
                {
                    "date_bought" : 1368540890,
                    "customer_id" : 9872345
                },
                {
                    "date_bought" : 1368537290,
                    "customer_id" : 9163893
                }
            ]
        }
    ]
}

I'm after output in the following format, that aggregates across all instances of each (unique) author across all daily dumps:

{
    "_id" : "Charles Dickens",
    "original_stock" : 253,
    "customers" : [
        {
            "date_bought" : 1368627290,
            "customer_id" : 9715923
        },
        {
            "date_bought" : 1368622358,
            "customer_id" : 9876234
        },
        etc...
    ]
}

I have written this map function...

function map() {
  for (var i in this.authors_who_sold_books)
  {
    author = this.authors_who_sold_books[i];
    emit(author.id, {customers: author.customers, original_stock: author.original_stock, num_sold: 1});
  }
}

...and this reduce function.

function reduce(key, values) {
  sum = 0
  for (i in values)
  {
    sum += values[i].customers.length
  }
  return {num_sold : sum};
}

However, this gives me the following output:

{
  "_id" : "Charles Dickens",
  "value" : {
    "customers" : [
      {
        "date_bought" : 1368627290,
        "customer_id" : 9715923
      },
      {
        "date_bought" : 1368622358,
        "customer_id" : 9876234
      },
    ],
    "original_stock" : 253,
    "num_sold" : 1
  }
}
{ "_id" : "JRR Tolkien", "value" : { "num_sold" : 3 } }
{
  "_id" : "JK Rowling",
  "value" : {
    "customers" : [
      {
        "date_bought" : 1368627290,
        "customer_id" : 9715923
      },
      {
        "date_bought" : 1368622358,
        "customer_id" : 9876234
      },
    ],
    "original_stock" : 183,
    "num_sold" : 1
  }
}
{ "_id" : "John Grisham", "value" : { "num_sold" : 2 } }

The even indexed documents have the customers and original_stock listed, but an incorrect sum of num_sold. The odd indexed documents only have the num_sold listed, but it is the correct number.

Could anyone tell me what it is I'm missing, please?

jimoleary jimoleary · Accepted Answer · 2013-05-24T21:53:58

Your problem is due to the fact that the format of the output of the reduce function should be identical to the format of the map function (see requirements for the reduce function for an explanation).

You need to change the code to something like the following to fix the problem, :

function map() {
  for (var i in this.authors_who_sold_books)
  {
    author = this.authors_who_sold_books[i];
    emit(author.id, {customers: author.customers, original_stock: author.original_stock, num_sold: author.customers.length});
  }
}

function reduce(key, values) {
  var result = {customers:[] , num_sold:0, original_stock: (values.length ? values[0].original_stock : 0)};
  for (i in values)
  {
    result.num_sold += values[i].num_sold;
    result.customers = result.customers.concat(values[i].customers);
  }
  return result;
}

I hope that helps.

Note : the change num_sold: author.customers.length in the map function. I think that's what you want

MongoDB MapReduce producing different results for each document

1 Answers