Less count of rows inserted to big query table than expected

0
votes

I have inserted 71 rows to day partitioned table. After streaming buffer workers processes there are only 48 rows in my table. I know that there are deduplication process, but the rows were different. I've tried to check how the rows look in streaming buffer and I've set condition to get results with _PARTITIONTIME set to NULL. I've got nothing.

I have no idea how is it working.

//Edit: I've inserted data using insertRows method from cloud-bigquery library. I'm selecting rows by using this query:

SELECT * FROM [table$20180216]
1
google-bigquery
How did you insert your data and how are you querying it? You can add this info to your post by editing it - VictorGGl
I've edited post and I've added this info to my post - jcala97
As a side note, it's recommended to use standard SQL: SELECT * FROM table WHERE _PARTITIONDATE = "2018-02-16" You can enable standard SQL clicking on the Show Options in the UI. More information: cloud.google.com/bigquery/docs/reference/standard-sql/… - Daria

1 Answers

2
votes

If you set _PARTITIONTIME to NULL, it selects 0 partitions and therefore 0 rows. Try querying the partition you inserted rows into by setting _PARTITIONDATE = "2018-02-15" (or change to the date you need).