Mongoose - RangeError: Maximum Call Stack Size Exceeded

Question

I am trying to bulk insert documents into MongoDB (so bypassing Mongoose and using the native driver instead as Mongoose doesn't support bulk insert of an array of documents). The reason I'm doing this is to improve the speed of writing.

I am receiving the error "RangeError: Maximum Call Stack Size Exceeded" at console.log(err) in the code below:

function _fillResponses(globalSurvey, optionsToSelectRegular, optionsToSelectPiped, responseIds, callback) {
  Response.find({'_id': {$in: responseIds}}).exec(function(err, responses) {
    if (err) { return callback(err); }

    if (globalSurvey.questions.length) {
      responses.forEach(function(response) {
        console.log("Filling response: " + response._id);
        response.answers = [];
        globalAnswers = {};
        globalSurvey.questions.forEach(function(question) {
          ans = _getAnswer(question, optionsToSelectRegular, optionsToSelectPiped, response);
          globalAnswers[question._id] = ans;
          response.answers.push(ans);
        });
      });
      Response.collection.insert(responses, function(err, responsesResult) {
        console.log(err);
        callback()
      });
    } else {
        callback();
      }
  });
} 

So similar to: https://stackguides.com/questions/24356859/mongoose-maximum-call-stack-size-exceeded

Perhaps it's something about the format of the responses array that Mongoose returns that means I can't directly insert using MongoDB natively? I've tried .toJSON() on each response but no luck.

I still get the error even with a very small amount of data but looping through and calling the Mongoose save on each document individually works fine.

EDIT: I think it is related to this issue: http://howtosjava.blogspot.com.au/2012/05/nodejs-mongoose-rangeerror-maximum-call.html

My schema for responses is:

var ResponseSchema = new Schema({
  user: {
    type: Schema.ObjectId,
    ref: 'User'
  },
  randomUUID: String,
  status: String,
  submitted: Date,
  initialEmailId: String,
  survey: String,
  answers: [AnswerSchema]
}); 

So, answers are a sub-document within responses. Not sure how to fix it though....

Answer 1

I was having this same issue and I started digging through the mongoose source code (version 3.8.14). Eventually it led me to this line within

• mongoose/node_modules/mongodb/lib/mongodb/collection/core.js -> insert(...) -> insertWithWriteCommands(...) ->

• mongoose/node_modules/mongodb/lib/mongodb/collection/batch/ordered.js -> bulk.insert(docs[i]) -> addToOperationsList(...) -> bson.calculateObjectSize(document, false);

  var bsonSize = bson.calculateObjectSize(document, false);

Apparently, this calls BSON.calculateObjectSize, which calls calculateObjectSize which then infinitely recurses. I wasn't able to dig that far in to what caused it, but figured that it may have something to do with the mongoose wrapper binding functions to the Schema. Since I was inserting raw data into mongoDB, once I decided to change the bulk insert in mongoose to a standard javascript object, the problem went away and bulk inserts happened correctly. You might be able to do something similar.

Essentially, my code went from

//EDIT: mongoose.model needs lowercase 'm' for getter method

var myModel = mongoose.model('MyCollection');
var toInsert = myModel();
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});

to

//EDIT: mongoose.model needs lowercase 'm' for getter method

var myModel = mongoose.model('MyCollection');
var toInsert = { //stuff in here 
   name: 'john',
   date: new Date()
};
var array = [toInsert];
myModel.collection.insert(array, {}, function(err, docs) {});

Answer 2

Confirmed, but not a bug. Model.collection.insert() bypasses Mongoose and so you're telling the node driver to insert an object that contains mongoose internals like $__, etc. The stack overflow is probably because bson is trying to compute the size of an object that references itself indirectly.

Long story short, use Document.toObject(), that's what its for: http://mongoosejs.com/docs/api.html#document_Document-toObject

Response.find({}).exec(function(err, responses) {
  if (err) { 
    return callback(err); 
  }

  if (true) {
    var toInsert = [];
    responses.forEach(function(response) {
      console.log("Filling response: " + response._id);
      response.answers = [];
      [{ name: 'test' }].forEach(function(ans) {
        response.answers.push(ans);
      });
      toInsert.push(response.toObject());
    });
    Response.collection.insert(toInsert, function(err, responsesResult) {
      console.log(err);
    });
  } else {
      callback();
    }
});

Also, the code you specified won't work even if you fix the stack overflow. Since you're trying to insert() docs that are already in the database, all the inserts will fail because of _id conflicts. You'd really be much better off just using a stream() to read the results one at a time and then save() them back into the db.

Answer 3

I have faced similar issue.

//manyvalues is array of objects
schema.methods.somemethod = function(manyvalues,callback) {
    this.model(collection).collection.insertMany(manyvalues,callback);
}

But this caused error [RangeError: Maximum call stack size exceeded]. So I have created new model from manyvalues and used it as below and it worked.

schema.methods.somemethod = function(manyvalues,callback){
     var list = JSON.parse(JSON.stringify(manyvalues));//created a new object.
     this.model(collection).collection.insertMany(list,callback);
}

The problem may be caused if manyvalues is changed internally.

Answer 4

guys! I've faced that weird error today. It happened because of I had a Schema with ref properties and tried to pass in create/update whole related document. I've changed argument to _id only and that did the trick. Works like a charm. I found the answer here (scroll down to February 21, 2013, 8:05 pm gustavohenke comment).

Answer 5

This also happens if there's a duplication of of the _id value. Most situations will be when you might create an new record from an existing record.

Deleting the _id and inserting the record and letting Mongoose/MongoDb take care of the creation of the id.

Answer 6

Check for circular references in the responses object. I Faced a similar issue due to circular references.

Answer 7

I had a similar problem, it was that I was querying a field that didn't exist in the schema using the $ne(other query operators may have a similar problem)

var TestSchema = new Schema({
  test:[]
});
...
models.Test.findOne({"test2": {$ne: "t"} })...

In the example above I am testing for test2 instead of test