Google Bigquery: Spark - Incompatible table partitioning specification

Question

While submitting a copy job from a temporary table that isn't partitioned to the final table that has partition by day, I recieve

cause:java.io.IOException: ErrorMessage: Incompatible table partitioning specification. Expects partitioning specification interval(type:day), but input partitioning specification is none;

Seeing another person with this issue this here

I am now facing the same issue, using the Spark saveAsNewAPIHadoopDataset which using the connector creates temp table then presumably using the copy command to load the data to the destination

Is there an eta on the fix Pavan?

Answer 1

We have recently made a change to support this (it will copy into the partition for the current date in this case). We expect the change to be released soon.