how bigquery backfill works? did it delete the old ones and reinsert again?

Question

When we run bigquery transfer (youtube) backfill.

How bigquery transfer backfill guarantee that no duplicated records are inserted?

It is not a database that you could do "if new insert, if old update". Bigquery transfer is delete and insert, right?

So bigquery transfer backfill would delete the old data of the backfill scheduled date and then re-insert the data?

I am trying to figure out when sometimes I get zero data but the transfer status is complete.

In my many times tests, it seems the old data was not deleted. But in one test, I did see the old data was deleted in backfill. (I could not re-produce it though).

Answer 1

BigQuery DTS backfills are indeed write truncate operations (delete and insert). It sounds like you resolved your own question: two transfers both backfilling to the same table may be causing the issue.