BigQuery: Why does Table Range Decorators return wrong result sometimes?

4
votes

I've been using the Table Range Decorators feature daily since May in order to only query the data from the last 7 days in some of my tables.

Since 2 weeks, I've noticed that sometimes some data is missing when I use that feature. For example, if do a query to get the results for the last 7 days (by adding "@-604800000--1" to table), some data will be missing as opposed to if I query on the whole table (without a table decorator).

I wonder what could explain this and if there is a fix coming soon to address this?

If this can help the BigQuery team, I've noticed that when using Table Decorators some data was missing for us for October 16th between around 16:00 and 20:00 UTC time.

For the BigQuery team here are 2 jobs ids where some data is missing: job_-xtL4PlIYhNjQ5weMnssvqDmd6U , job_9ASNxqq_swjCd1eMmiQ6SmPpxlQ

and 1 job id where data is correct(without decorators): job_QbcRwYGbQv0BZdHreQEvRlYh-mM

1
google-bigquery

1 Answers

2
votes

This is a known issue with table decorators containing a time range. Due to a bug in BigQuery, it is possible for certain time ranges to omit data that should be included within the time range.

We're working on a fix and plan to have it released next week. After this fix is deployed time range decorators should again work as expected.