Unmarshalling BSON data using Gobson/Mgo

Question

I am receiving different BSON documents, an example seen here:

{ "group" : { "id": ObjectId("11a123456bc345d452d32c0b5"), "name": "SomeName" }, "count1": 6, "count2": 33, "totalCount": 39 }

I am trying to use mgo/gobson to unmarshal this data into a map[string]interface{}, but keep getting an error. The unmarshalling code is

var record map[string]interface{}

err := bson.Unmarshal(data, &record)

This resulted in a error saying the document was corrupted. Looking into the code I saw that it expected the length of the byte array to be the first few bytes so I added code similar to what is found in the tests.

rec := make([]byte, len(data)+6)
binary.LittleEndian.PutUint32(rec, uint32(len(rec)))
copy(rec[4:], data)

This got me past the corrupt document error, but now I am getting

Unknown element kind (0x7B)

What am I doing wrong that I can't easily unmarshal the above BSON document into a map? Is there a problem with the BSON document itself? I am not the creator of the BSON document, but am able to provide feedback to them if necessary.

This sounds like a problem with the BSON document, not your code. Where is the document coming from? — 3rf
The document is provided by another team. I am responsible for pulling and ingesting the data into the system that I work with. — ryknow
I spoke with the other team and was told that they have a script for querying MongoDB and dumping the results into a file and that is what I am pulling. — ryknow
Does your BSON literally look like the human-readable { "group" : { "id": ObjectId("11a123456bc345d452d32c0b5"), "name": "SomeName" }, "count1": 6, "count2": 33, "totalCount": 39 } or is it in binary form? — 3rf
If they want to get you BSON, they should be using the mongodump tool. I would double-check with them to make sure. It would be really helpful if you could outline their workflow here. — 3rf

foecum foecum · Accepted Answer · 2016-08-25T10:10:22

I tried out your code above and I managed to get it to "work". The reason you are getting

Unknown element kind (0x7B)

is because there is no element of kind 0x7B in the BSON specification. See here http://bsonspec.org/spec.html. The particular unknow element I was getting was the byte value for the first curly bracket({) in your document

{ "group" : { "id": ObjectId("11a123456bc345d452d32c0b5"), "name": "SomeName" }, "count1": 6, "count2": 33, "totalCount": 39 }

after tweaking the code a bit I was able to get it working with out the unknown kind error.

package main

import (
    "encoding/binary"
    "fmt"

    "gopkg.in/mgo.v2/bson"
)

func main() {
    bsonData := `{ "group" : { "id": ObjectId("11a123456bc345d452d32c0b5"), "name": "SomeName" }, "count1": 6, "count2": 33, "totalCount": 39 }`

    var record map[string]interface{}

    rec := make([]byte, len(bsonData)+6)
    binary.LittleEndian.PutUint32(rec, uint32(len(rec)))
    copy(rec[4:], bsonData)
    //Adding a new line to the front of the rec byte variable seems to do the trick
    r := []byte("\n")
    rec[4] = r[0]
    err := bson.Unmarshal(rec, &record)
    if err != nil {
        panic(err)
    }
    fmt.Println(record)
}

I also found this answer to be of great use https://stackoverflow.com/a/33129119/2627768

Unmarshalling BSON data using Gobson/Mgo

1 Answers