Unmarshal CSV record into struct in Go

17
votes

The problem how to automatically deserialize/unmarshal record from CSV file into Go struct.

For example, I have 

type Test struct {
  Name string
  Surname string
  Age int
}

And CSV file contains records

John;Smith;42
Piter;Abel;50

Is there an easy way to unmarshal those records into struct except by using "encoding/csv" package for reading record and then doing something like

record, _ := reader.Read()
test := Test{record[0],record[1],atoi(record[2])}
4
csvgounmarshalling
Nope, the pattern you have is the way to go. (I bet record, _ := Read() was just to get concise example code here, but do handle the error in your real code, or it'll bite you when the program someday misbehaves and you don't know why.) – twotwotwo
Ehh... I hoped there is some package that will use reflection like xml/json unmarshalling. And of course ignoring error just to minimize source of example by skipping non-relevant code. – Valentyn Shybanov
I wonder why they didn't write such a package. Might be fun to write one yourself. – MatrixFrog
I know this is many years later, I am just wondering what would be a good use case to do this? I feel like Go is strongly typed and in most cases you would know the schema of the csv up-front, so testing the type of each field in each row would be slow/redundant. Perhaps a tool that is designed to infer and suggest schema to users? If it is to avoid boilerplate or hardcoding type conversions perhaps separating the schema to a struct with a method to convert it is a solution? – Davos

4 Answers

19
votes

There is gocarina/gocsv which handles custom struct in the same way encoding/json does. You can also write custom marshaller and unmarshaller for specific types.

Example:

type Client struct {
    Id      string `csv:"client_id"` // .csv column headers
    Name    string `csv:"client_name"`
    Age     string `csv:"client_age"`
}

func main() {
    in, err := os.Open("clients.csv")
    if err != nil {
        panic(err)
    }
    defer in.Close()

    clients := []*Client{}

    if err := gocsv.UnmarshalFile(in, &clients); err != nil {
        panic(err)
    }
    for _, client := range clients {
        fmt.Println("Hello, ", client.Name)
    }
}
14
votes

Seems I've done with automatic marshaling of CSV records into structs (limited to string and int). Hope this would be useful.

Here is a link to playground: http://play.golang.org/p/kwc32A5mJf

func Unmarshal(reader *csv.Reader, v interface{}) error {
    record, err := reader.Read()
    if err != nil {
        return err
    }
    s := reflect.ValueOf(v).Elem()
    if s.NumField() != len(record) {
        return &FieldMismatch{s.NumField(), len(record)}
    }
    for i := 0; i < s.NumField(); i++ {
        f := s.Field(i)
        switch f.Type().String() {
        case "string":
            f.SetString(record[i])
        case "int":
            ival, err := strconv.ParseInt(record[i], 10, 0)
            if err != nil {
                return err
            }
            f.SetInt(ival)
        default:
            return &UnsupportedType{f.Type().String()}
        }
    }
    return nil
}

I'll try to create github package is someone needs this implementation.

1
votes

You could bake your own. Perhaps something like this:

package main

import (
    "fmt"
    "strconv"
    "strings"
)

type Test struct {
    Name    string
    Surname string
    Age     int
}

func (t Test) String() string {
    return fmt.Sprintf("%s;%s;%d", t.Name, t.Surname, t.Age)
}

func (t *Test) Parse(in string) {
    tmp := strings.Split(in, ";")
    t.Name = tmp[0]
    t.Surname = tmp[1]
    t.Age, _ = strconv.Atoi(tmp[2])
}

func main() {

    john := Test{"John", "Smith", 42}
    fmt.Printf("john:%v\n", john)

    johnString := john.String()
    fmt.Printf("johnString:%s\n", johnString)

    var rebornJohn Test
    rebornJohn.Parse(johnString)
    fmt.Printf("rebornJohn:%v\n", rebornJohn)

}
1
votes

Using csvutil it is possible to give column header see example.

In your case, this could be :

package main

import (
    "encoding/csv"
    "fmt"
    "io"
    "os"

    "github.com/jszwec/csvutil"
)

type Test struct {
    Name    string
    Surname string
    Age     int
}

func main() {
    csv_file, _ := os.Open("test.csv")
    reader := csv.NewReader(csv_file)
    reader.Comma = ';'

    userHeader, _ := csvutil.Header(Test{}, "csv")
    dec, _ := csvutil.NewDecoder(reader, userHeader...)

    var users []Test
    for {
        var u Test
        if err := dec.Decode(&u); err == io.EOF {
            break
        }
        users = append(users, u)
    }

    fmt.Println(users)
}