1
votes

I'm trying to figure out if there a possibility to config incremental refresh for the old data which could be updated.

E.g. we have initial table data (Azure SQL DB as a source):

audit_date val updated_ts
2017-07-01 00:00:00.000 10 2017-07-01 00:00:00.000
2018-06-01 00:00:00.000 20 2018-06-01 00:00:00.000
2020-12-20 00:00:00.000 30 2020-12-20 00:00:00.000

and incremental refresh properties as below (audit_date is non-changable filtered column):

enter image description here

After data is published and refreshed it reflects whole source data (Power BI Pro):

enter image description here

Then we update some val column value (updated_ts receives current datetime value) and get below rows on the source:

audit_date val updated_ts
2017-07-01 00:00:00.000 10 2017-07-01 00:00:00.000
2018-06-01 00:00:00.000 555 2020-12-24 13:01:02.000
2020-12-20 00:00:00.000 30 2020-12-20 00:00:00.000

After refresh we still see initial values:

enter image description here

Does it mean that incremental refresh does not catch changes of old data or my understanding is wrong? Perhaps there is another way exists for such data changes handling?

1
Do you have Power BI Premium and is this workspace running on it? If so you could build your own partitioning and incremental processing tool.GregGalloway
so far I have only Pro licenseAlex

1 Answers

0
votes

You've limited your rows to refresh to the last 30 days according to the audit_date column. Thus the row in question outside of the scope you've set.

The official documentation makes a similar remark:

Filter date column updates

The filter on the date column is used to dynamically partition the data into ranges in the Power BI service. Incremental refresh isn't designed to support cases where the filtered date column is updated in the source system. An update is interpreted as an insertion and a deletion, not an actual update. If the deletion occurs in the historical range and not the incremental range, it won't get picked up. This can cause data refresh failures due to partition-key conflicts.