Programmatically fill golang struct

Question

I have a file with many types of data record which I need to parse into structs.

I'd be grateful to learn of a idiomatic way -- if it exists -- of filling structs by record type. Something like python's namedtuple(*fields) constructor.

package main

import (
    "fmt"
    "strconv"
    "strings"
)

type X interface{}

type HDR struct {
    typer, a string
    b        int
}

type BDY struct {
    typer, c string
    d        int
    e        string
}

var lines string = `HDR~two~5
BDY~four~6~five`

func sn(s string) int {
    i, _ := strconv.Atoi(s)
    return i
}

func main() {
    sl := strings.Split(lines, "\n")
    for _, l := range sl {
        fields := strings.Split(l, "~")
        var r X
        switch fields[0] {
        case "HDR":
            r = HDR{fields[0], fields[1], sn(fields[2])} // 1
        case "BDY":
            r = BDY{fields[0], fields[1], sn(fields[2]), fields[3]} // 2
        }
        fmt.Printf("%T : %v\n", r, r)
    }
}

I'm specifically interested to learn if lines marked // 1 and // 2 can be conveniently replaced by code, perhaps some sort of generic decoder which allows the struct itself to handle type conversion.

Using reflection will make your code much harder to read. The reflect package is for rare use cases, mostly to implement things like the csv library that you pointed to. Struct tags are read with the reflect package in that case. You do not want to use it in reality though, why would you complicate things like that? As I said, your code is easy to understand, break up the lines, split at ~ and parse the thing. Straight forward, easy to understand, easy to change, it has no dependencies. The problem with the csv library is that it does not make the code shorter or better to read. ... — gonutz
... Looking at the csv library I see a zillion .go files, adding this complexity for such an easy problem will not benefit you. — gonutz
Thanks for your comments, @gonutz. I'm not sure what style you recommend (you seem to use single char variable names yourself!). Also you haven't qualified why you believe reflection is a bad idea. As previously mentioned in my edited comment, it would be good to know whether decoding (eg using github.com/jszwec/csvutil) is a better route? It's also not clear why you are proscribing complexity per se -- or perhaps that is just a "note to self"? In which case, thanks for sharing. Otherwise, I think you just hit a "nul points" for your comment -- on the Wogan scale! — rorycl
I don't know what that means. Please read my other comments for the qualification of why reflection is not the way to go. — gonutz
Trying to rewrite your code into an answer I realize that I do not understand what the code is supposed to do. What is the purpose of having these structs in the same interface{} form? Do you want to put them into an array? Why not treat them separately anyway? You seem to have a header and a body type. Why would you even want to treat them as the same thing anyway? — gonutz

Cerise Limón Cerise Limón · Accepted Answer · 2019-10-22T00:10:17

Use the reflect package to programmatically set fields.

A field must be exported to be set by the reflect package. Export the names by uppercasing the first rune in the name:

type HDR struct {
    Typer, A string
    B        int
}

type BDY struct {
    Typer, C string
    D        int
    E        string
}

Create a map of names to the type associated with the name:

var types = map[string]reflect.Type{
    "HDR": reflect.TypeOf((*HDR)(nil)).Elem(),
    "BDY": reflect.TypeOf((*BDY)(nil)).Elem(),
}

For each line, create a value of the type using the types map:

for _, l := range strings.Split(lines, "\n") {
    fields := strings.Split(l, "~")
    t := types[fields[0]]
    v := reflect.New(t).Elem()
    ...
}

Loop over the fields in the line. Get the field value, convert the string to the kind of the field value and set the field value:

    for i, f := range fields {
        fv := v.Field(i)
        switch fv.Type().Kind() {
        case reflect.String:
            fv.SetString(f)
        case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
            n, _ := strconv.ParseInt(f, 10, fv.Type().Bits())
            fv.SetInt(n)
        }
    }

This is a basic outline of the approach. Error handling is notabling missing: the application will panic if the type name is not one of the types mentioned in types; the application ignores the error returned from parsing the integer; the application will panic if there are more fields in the data than the struct; the application does not report an error when it encounters an unsupported field kind; and more.

Run it on the Go Playground.

Programmatically fill golang struct

1 Answers