Gorm always return structs with nil values

Question

I'm building a Go web API with Gorm as the ORM for Postgresql databases in Amazon RDS. Problem is Gorm always gives back a slice of structs which values are all nil, although the database is already populated with data. The number of structs in the slice are proper depending on the LIMIT I gave.

I've also tried to directly query the SQL using database/sql builtin package, manually insert the variable inside rows.Next() loop and it works with no problem. I've tried this with 3 different tables with 3 different structs (obviously) and the result are all the same. So I guess it's a problem in the Gorm side. One of the table is given as example below.

Struct structure

type gameCenterLog struct {
    tm      time.Time
    seq     int
    uid     int
    partner int
    token   sql.NullString
    bounty  int
    path1   sql.NullString
    path2   sql.NullString
    path3   sql.NullString
    action  sql.NullString
    value1  sql.NullString
    value2  sql.NullString
    value3  sql.NullString
    value4  sql.NullString
    value5  sql.NullString
}
func (log *gameCenterLog) TableName() string {
    return "gamecenter_log"
}

Example of the Gorm query

func testGorm() {
    connString := fmt.Sprintf("host=%v port=%v user=%v password=%v dbname=%v", host, port, user, pass, schema)
    db, err := gorm.Open("postgres", connString)
    if err != nil {
        panic(err.Error())
    }
    defer db.Close()

    var logs []gameCenterLog

    today := time.Now().UTC()
    lastWeek := time.Now().Add(-7 * 24 * time.Hour).UTC()

    db.Debug().Where("tm BETWEEN ? AND ?", lastWeek, today).Order("tm desc").Limit(limit).Find(&logs)
    fmt.Printf("Error: %+v\n", db.Error)
    for _, result := range logs {
        fmt.Printf("%+v\n", result)
    }
}

Gorm Result

[2018-08-15 09:57:15]  [1429.58ms]  SELECT * FROM "gamecenter_log"  WHERE (tm BETWEEN '2018-08-08 02:57:14' AND '2018-08-15 02:57:14') ORDER BY tm desc LIMIT 10  
[10 rows affected or returned ] 
Error: <nil>
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:0 loc:<nil>} seq:0 uid:0 partner:0 token:{String: Valid:false} bounty:0 path1:{String: Valid:false} path2:{String: Valid:false} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}

Notice all results are nill and there are no error in the result...

Now with database/sql

func testSQL() {
    connString := fmt.Sprintf("host=%v port=%v user=%v password=%v dbname=%v", host, port, user, pass, schema)
    db, err := sql.Open("postgres", connString)
    if err != nil {
        panic(err.Error())
    }
    defer db.Close()

    today := time.Now().UTC()
    lastWeek := time.Now().Add(-7 * 24 * time.Hour).UTC()

    query := fmt.Sprintf("SELECT * FROM \"%v\" WHERE (%v BETWEEN '%d-%02d-%02d %02d:%02d:%02d' AND '%d-%02d-%02d %02d:%02d:%02d') ORDER BY %v LIMIT %v",
        "gamecenter_log", "tm",
        lastWeek.Year(), lastWeek.Month(), lastWeek.Day(), lastWeek.Hour(), lastWeek.Minute(), lastWeek.Second(),
        today.Year(), today.Month(), today.Day(), today.Hour(), today.Minute(), today.Second(),
        "tm desc", limit)
    fmt.Println(query)
    rows, err := db.Query(query)
    if err != nil {
        panic(err.Error())
    }

    results := []gameCenterLog{}
    for rows.Next() {
        result := gameCenterLog{}
        err := rows.Scan(
            &result.tm, &result.seq, &result.uid,
            &result.partner, &result.token, &result.bounty,
            &result.path1, &result.path2, &result.path3, &result.action,
            &result.value1, &result.value2, &result.value3, &result.value4, &result.value5,
        )
        if err != nil {
            panic(err.Error())
        }
        results = append(results, result)
    }
    for _, result := range results {
        fmt.Printf("%+v\n", result)
    }
}

Here I manually append the result struct to the results slice

Raw SQL Result

SELECT * FROM "gamecenter_log" WHERE (tm BETWEEN '2018-08-08 03:03:41' AND '2018-08-15 03:03:41') ORDER BY tm desc LIMIT 10
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846269 uid:16199265 partner:1 token:{String:1534274713356-bb09968a07 Valid:true} bounty:3 path1:{String:result Valid:true} path2:{String:Win Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846278 uid:16060021 partner:15 token:{String:1534272218332-2a6a0a3263 Valid:true} bounty:2 path1:{String:result Valid:true} path2:{String:Win Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String:102012 Valid:true} value2:{String:68368886 Valid:true} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846275 uid:14565958 partner:6 token:{String:1534277021722-d2f5a72098 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:PlayStart Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846279 uid:16060021 partner:15 token:{String:1534272218332-2a6a0a3263 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:PlayEnd Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String:68368886 Valid:true} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846268 uid:16199265 partner:1 token:{String:1534274713356-bb09968a07 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:PlayEnd Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String: Valid:false} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846270 uid:16199265 partner:1 token:{String:1534274713356-bb09968a07 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:Custom Valid:true} path3:{String: Valid:false} action:{String:PemulaGameWin Valid:true} value1:{String:9700600 Valid:true} value2:{String:20000 Valid:true} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846280 uid:7627835 partner:15 token:{String:1534256412228-80b0aacd5b Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:PlayStart Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String:66729274 Valid:true} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899021 loc:0xc420113c20} seq:1534276846277 uid:10212903 partner:4 token:{String:1534270203280-7875331136 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:Custom Valid:true} path3:{String: Valid:false} action:{String:ExpertLose Valid:true} value1:{String:+9.50K Valid:true} value2:{String:5000 Valid:true} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899020 loc:0xc420113c20} seq:1534276846208 uid:13016587 partner:15 token:{String:1534265134142-ed98a488f1 Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:PlayStart Valid:true} path3:{String: Valid:false} action:{String: Valid:false} value1:{String:67935379 Valid:true} value2:{String: Valid:false} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}
{tm:{wall:0 ext:63669899020 loc:0xc420113c20} seq:1534276846265 uid:15125780 partner:1 token:{String:1534267652681-cfb8846eae Valid:true} bounty:0 path1:{String:track Valid:true} path2:{String:Custom Valid:true} path3:{String: Valid:false} action:{String:AhliGameLose Valid:true} value1:{String:-1000000 Valid:true} value2:{String:1000000 Valid:true} value3:{String: Valid:false} value4:{String: Valid:false} value5:{String: Valid:false}}

The result comes out no problem.

Okay so changing the case indeed fixed the issue... I didn't think of exportable variable. Thanks for the answer! Please put the answer on an answer btw, rather than comment — Christian Lim
I'm pretty sure this is a duplicate (hence the comments) but my search abilities are lacking today. — mu is too short
I've been searching as well but couldn't find the answer. Maybe someone else with totally different problems and different symptoms have the same solution as mine. I guess it's because the title only says the symptoms and not the solution. — Christian Lim

mu is too short mu is too short · Accepted Answer · 2018-08-15T04:36:00

All the fields in your gameCenterlog struct are lower case so they're not exported. Non-exported fields are invisible to reflection so they're invisible to Gorm.

If you capitalize the fields:

type gameCenterLog struct {
    Tm      time.Time
    Seq     int
    Uid     int
    ...

then Gorm will be able to see them give values. Gorm should be able to figure out the mapping from the column names in PostgreSQL to the struct fields in Go but you can use gorm:"column:..." struct tags if it doesn't.

Gorm always return structs with nil values

1 Answers