How to escape $ while inserting inside mongoDB

1
votes

I have a large object which I have no control over, fields might vary a lot:

Error ==>> { MongoError: The dollar ($) prefixed field '$' in 'value.external.originalRequest.rawRequest.ns2:LoadData.Orders.0.Order.0.TotalThirdPartyCharge.0.CurrencyValue.0.$' is not valid for storage. 0|lt-check | at Function.MongoError.create (/home/ops/lt-checkin-api/node_modules/mongodb-core/lib/error.js:31:11) 0|lt-check | at /home/ops/lt-checkin-api/node_modules/mongodb-core/lib/connection/pool.js:497:72 0|lt-check | at authenticateStragglers (/home/ops/lt-checkin-api/node_modules/mongodb-core/lib/connection/pool.js:443:16) 0|lt-check | at Connection.messageHandler (/home/ops/lt-checkin-api/node_modules/mongodb-core/lib/connection/pool.js:477:5) 0|lt-check | at Socket. (/home/ops/lt-checkin-api/node_modules/mongodb-core/lib/connection/connection.js:333:22) 0|lt-check | at Socket.emit (events.js:159:13) 0|lt-check | at addChunk (_stream_readable.js:265:12) 0|lt-check | at readableAddChunk (_stream_readable.js:252:11) 0|lt-check | at Socket.Readable.push (_stream_readable.js:209:10) 0|lt-check | at TCP.onread (net.js:608:20) 0|lt-check | name: 'MongoError', 0|lt-check | message: 'The dollar ($) prefixed field \'$\' in \'value.external.originalRequest.rawRequest.ns2:LoadData.Orders.0.Order.0.TotalThirdPartyCharge.0.CurrencyValue.0.$\' is not valid for storage.', 0|lt-check | ok: 0, 0|lt-check |
errmsg: 'The dollar ($) prefixed field \'$\' in \'value.external.originalRequest.rawRequest.ns2:LoadData.Orders.0.Order.0.TotalThirdPartyCharge.0.CurrencyValue.0.$\' is not valid for storage.', 0|lt-check | code: 52, 0|lt-check |
codeName: 'DollarPrefixedFieldName' }

Here the key causing the error:

"rawRequest": {
                        "ns2:LoadData": {
                            "$": {
                                "xmlns:ns2": "http://schemas.3gtms.com/tms/v1/tns"
                            },
                            "BatchInfo": [
                                {
                                    "$": {
                                        "xmlns": ""
                                    },

As there are multiple keys starting with $.

Is there a setting in mongoose or mongoDB itself to insert anything without validating the request coming in.

2
node.jsmongodbmongoosekoa
Why can't you do it in node before inserting into mongo? – Anirudh Simha
I am not sure about the keys and moreover is there a good method to escape all the keys having $. @AnirudhSimha – Black Mamba
check docs.mongodb.com/manual/reference/limits for reserved keywords.If it is a linux env its /\. "$ – Anirudh Simha
Was just looking around and found this: npmjs.com/package/mongo-escape – Anirudh Simha
Great let me check if it helps – Black Mamba

2 Answers

0
votes

Was going to create a new plugin but @Anirudh helped me find this package in the comment

https://www.npmjs.com/package/mongo-escape

To use it, just before your insert query escape you object using the above package.

var escapeForMongo = require('mongo-escape').escape;
userInput = escapeForMongo({
  'foo': 'bar',
  'ba.z': {
    '$in': 'quz'
  }
})
0
votes

function escapeMongo(obj) {
    return JSON.parse(
        JSON.stringify(obj),
        function(p, v) {
            if ((p[0] === '$') || (p.includes('.'))) {
                this[p.replace(/\$/g, '\uFF04').replace(/\./g, '\uFF0E')] = v;
                return;
            }
            return v;
        }
    )
}

userInput = {
    'foo': 'bar',
    'ba.z': {
        '$in': 'quz'
    }
}
console.log(userInput)
console.log(escapeMongo(userInput))

You can use JSON.parse to process a JSON string before creating object.