How do I view the full path of an Airflow DAG?

0
votes

In the Airflow web interface I can view a list of DAGs. If I click on one of those DAGs I can see different views of the DAG or the code for the DAG, and in the details tab I can see the name of the DAGs file, but not the path.

I've also tried airflow list-dags but that doesn't provide the information I'm looking for either.

There is a DAG I can see in the web interface but I can't find where it is in the filesystem.

Is there a way to find the location of a DAG from the Airflow web interface or the Airflow CLI?

4
airflow
You can see filepath details under "details" tab in UI. – naga satish

4 Answers

5
votes

On the main DAGs list in the ui, there is a button to the left of the DAG pause toggle. It resolves to:

https://<airflow.blah.com>/admin/dagmodel/edit/?id=<dag_id>

The Fileloc will show you where airflow thinks your DAG file is. However...

  • it can be wrong if you import other local files into you DAG file the
  • the file may not be there anymore, this info is stored in the db
0
votes

i don't think so, but you should just search your dags_folder from your airflow.cfg to find where dags are created.

additionally, the logs for airflow-webserver tell you exactly which files/dags are being imported, assuming you have some logging on.

0
votes

Especially if you do development and change your DAGs / delete the DAG files, the entries will often persist in the backend DB and be visible in the UI. Most of the time, there isn't actually any DAG file anymore.

I'd recommend cleaning the DAGs from the database, using something like:

import sys
from airflow.hooks.postgres_hook import PostgresHook

dag_input = sys.argv[1]
hook=PostgresHook( postgres_conn_id= "airflow_db")

for t in ["xcom", "task_instance", "sla_miss", "log", "job", "dag_run", "dag" ]:
  sql="delete from {} where dag_id='{}'".format(t, dag_input)
  hook.run(sql, True)

(Based on an answer from Airflow: how to delete a DAG?)

0
votes

Airflow by default provides some sample DAGs. The DAGs are kept under the example_dags folder.

Path of the DAGs: lib/python2.7/site-packages/airflow/example_dags