Can you monitor the progress of an incremental refresh in Tableau programmatically?

Question

I have looked at the TDE (Tableau Data Extract) API and it seems like it is only used to create and edit Tableau Data Extracts but not to monitor progress.
I have also looked at tabcmd and it seems like unless you are doing a full refresh, using the --synchronous command, then there is no way to monitor the refresh unless you are logged into Tableau Server.
I even looked at connecting to the Postgres DB Tableau uses to see if I can monitor the refresh from _background_tasks, async_jobs, background_jobs, extracts, extract_sessions, or historical_events table but I still couldn't find a way to monitor the incremental refresh progress.

Am I missing something? I need to monitor the progress of an incremental refresh so when it finishes another process kicks off. Is there anyway to do this?

jimjim jimjim · Accepted Answer · 2016-12-16T14:25:23

I don't know how I over looked this, but I did. There is a column called job_name in the _background_tasks table. I thought "Refresh Extracts" was the only indicator, but there is also a value called "Increment Extracts". Here is a Python script to find those the incremental refresh jobs with whether they succeeded, their progress, the name of the extract, and the time they completed once they are finished. I'm using Python 2.7. I hope someone finds this useful.

import psycopg2


try:
    conn = psycopg2.connect("dbname='database' user='user' host='localhost' password='password' "
                        "port='portnumber'")
    cur = conn.cursor()
    cur.execute("SELECT finish_code, progress, title, completed_at FROM _background_tasks where job_name = 'Increment "
            "Extracts' and POSITION('my_incremental_extract' IN title) > 0")

    query_result = cur.fetchall()
    for row in query_result:
        if row[0] == 0:
            print 'Finished succesfully'
        else:
            print 'Failed'
        print 'Progress: ' + str(row[1])
        print 'Job: ' + row[2]
        print 'Time completed: ' + str(row[3])
        print '*******************************************new record*****************************************' 
except Exception as tb_ex:
    print tb_ex

Can you monitor the progress of an incremental refresh in Tableau programmatically?

1 Answers