Unmarshal embedded field pointer panics when field implements UnmarshalJSON

Question

I have a struct that embeds an embedded pointer to another struct. When I use the default json.Unmarshal behavior, it works perfectly. But when I implement UnmarshalJSON for the embedded struct's type but not the outer struct, then go panics with null pointer dereference.

If I implement UnmarshalJSON for the outer struct type as well, then it works. However, the outer struct has many fields that I would rather not have to manually unmarshal.

Why does implementing UnmarshalJSON on one and not the other cause a panic?
Is there a way to get it work without implemented UnmarshalJSON for the outer type?
If not, is there a simpler/less manual way to implement UnmarshalJSON for the outer type?

Note: There is a question with a similar title, "json.Unmarshal fails when embedded type has UnmarshalJSON ", but the issue there is different from mine.

tl;dr: The rest of this question is just a lengthy example of the above.

Base Example

(play.golang.org version of example)

The two structs, one with an embedded field pointer to the other:

(Simplified for example -- this doesn't really need its own UnmarshalJSON but it demonstrates the problem.)

type Obj struct {
    X int `json:"x"`
}

type Container struct {
    *Obj
    Y int `json:"y"`
}

Invoking unmarshal:

func main() {
    b := []byte(`{"x": 5, "y": 3}`)
    c := &Container{}
    err := json.Unmarshal(b, c)
    if err != nil {
        fmt.Printf("error ummarshalling json: %+v\n", err)
        return
    }
    fmt.Printf("unmarshalled: %+v --> %+v\n", c, c.Obj)
}

Without implementing any UnmarshalJSON funcs, this works fine:

unmarshalled: &{Obj:0x416080 Y:3} --> &{X:5}

Panic

But, if I add UnmarshalJSON to the embedded Obj type only, then the program panics, as the json.Unmarshal call passes a nil pointer when it tries to unmarshal *Obj.

func (o *Obj) UnmarshalJSON(b []byte) (err error) {
    m := make(map[string]int)
    err = json.Unmarshal(b, &m)
    if err != nil {
        return nil
    }
    o.X = m["x"] // the line indicated by panic
    return nil
}

Output:

panic: runtime error: invalid memory address or nil pointer dereference
[...]
main.(*Obj).UnmarshalJSON(0x0, 0x416030, 0x10, 0x10, 0x0, 0x0)
    /tmp/sandbox185809294/main.go:18 +0x130
[...]

Question: Why does it panic here but not with the default unmarshal behavior? I'd think that if a nil *Obj is being passed here, then the default behavior also passes around a nil pointer...

Fixing the panic

It no longer panics when I implement UnmarshalJSON for the outer Container type:

func (c *Container) UnmarshalJSON(b []byte) (err error) {
    m := make(map[string]int)
    err = json.Unmarshal(b, &m)
    if err != nil {
        return err
    }
    c.Obj = &Obj{X: m["x"]}
    c.Y = m["y"]
    return nil
}

But unmarshalling Container manually this way gets tedious if both the real Container and real Obj have more fields than this, each with different types.

Question: Is there a simpler way to prevent this panic?

Adrian Adrian · Accepted Answer · 2018-12-31T17:12:32

Because the default behavior checks for nil and your custom unmarshaller does not. You need some logic in your UnmarshalJSON to check if o is nil and behave appropriately, instead of assuming o is not nil (by trying to access one of its fields), thereby triggering a panic.

func (o *Obj) UnmarshalJSON(b []byte) (err error) {
    if o == nil {
        return nil // maybe? What do you want to happen in this case?
    }
    m := make(map[string]int)
    err = json.Unmarshal(b, &m)
    if err != nil {
        return nil
    }
    o.X = m["x"] // the line indicated by panic
    return nil
}

Also just for future reference, your *Obj field is not an "anonymous field", it is an embedded field: https://golang.org/ref/spec#Struct_types

Unmarshal embedded field pointer panics when field implements UnmarshalJSON

Base Example

Panic

Fixing the panic

1 Answers