Insert timestamp into bigquery table using pandas

Question

I have a table in Google bigquery, where a column is set to datatype timestamp.

I have to insert data using to_gbq function of pandas.

If I set the datatype to string instead of timestamp the data is loaded into the table.

But I want the column to be timestamp datatype.

How can the column type of dataframe converted to timestamp compatible with Google bigquery.

Error

Table schema

iDrwish iDrwish · Accepted Answer · 2019-09-11T11:15:27

Timestamp is stored as int64 datatype, or int, so if you would like to format it in such a way, you can store cast your column to this datatype in pandas.

import numpy as np
import pandas as pd
df.COLUMN_OBJECT = df.COLUMN_OBJECT.astype(np.int64)
df.COLUMN_DATETIME = df.COLUMN_DATETIME.apply(lambda x: x.timestamp).astype(np.int64)

You can have the timestamp in seconds, ms, or ns.

Insert timestamp into bigquery table using pandas

1 Answers