Custom scanner for structs in Go

Question

I am writing a sort of a REST-like API in Go. The database I have is legacy, so I don't control the field names or the structure or for that matter anything. I am abstracting database access methods in a separate package called datastore. The code I have looks like this (skipping all the error handling etc.)

type Datastore struct {}

type Object struct {
  id uint
  name string
  ... zillion more fields here
}

func (Datastore) ObjectList() {
  var objects *[]Object
  db, _ := sqlx.Open("postgres", conn_info)
  rows, _ := sqlx.Queryx("SELECT * FROM object_table")
  defer rows.Close()
  for rows.Next() {
    var obj Object
    rows.Scan(&obj.id, &obj.name)
    objects = append(objects, obj)
  }
  return objects
}

The problem I am currently having is that the object table has dozens and dozens of fields. Some I care about, but some I do not. Some are named the same as Object and some are not. Eventually, I will need to support most of them, but I am working on a proof of concept first. It seems that the Scan code fails if it finds more fields in the row than are present in the arguments for Scan(). I could list the fields I scan for in the query select id, name from object_table, but 1. it makes the code extremely ugly (SQL doesn't get formatted right by gofmt) 2. it adds another place I need to edit when I want to support another field

Is there any way to implement a custom scanner interface that would take the rows object, load some data out of it into a struct and ignore the rest?

You should make it clear you are using sqlx in your question, don't make people try to guess what is going on. Furthermore that's the way the API works. You need 3 fields, scans takes 3 values, period there is no way around it. If you need another behavior use another method, like Get or Select . — mpm
I would sugest you may try an ORM like github.com/jinzhu/gorm . — Sarath Sadasivan Pillai
@mpm Get() and Select() run into the same problem. What do you mean "there is no way around it"? There is always a way around it. It might be implementing my own scan, but I cannot seems to figure out how. — Mad Wombat
@Sarathsp yeah, I considered gorm, but the more complex queries seem to be awkward to implement in it — Mad Wombat
@MadWombat My point is that ultimately you are limited by sql/db API, that's how this API works. So no. Given a number of columns X you need to provide X arguments in Scan. Period . — mpm

mpm mpm · Accepted Answer · 2016-12-24T06:13:36

You are already using sqlx so just use *DB.Unsafe() before querying :

var db *sqlx.DB

// exactly the same as the built-in
db = sqlx.Open("sqlite3", ":memory:")
var p Person
udb := db.Unsafe()
err = udb.Get(&p, "SELECT * FROM person, place LIMIT 1;")

Custom scanner for structs in Go

1 Answers