BashOperator doen't run bash command apache airflow

Question

I was running the tutorial of airflow. The content in the tutorial.py is as follows:

"""
Code that goes along with the Airflow located at:
http://airflow.readthedocs.org/en/latest/tutorial.html
"""
from airflow import DAG
from airflow.operators.bash_operator import BashOperator
from datetime import datetime, timedelta


default_args = {
    'owner': 'airflow',
    'depends_on_past': False,
    'start_date': datetime(2015, 6, 1),
    'email': ['[email protected]'],
    'email_on_failure': False,
    'email_on_retry': False,
    'retries': 1,
    'retry_delay': timedelta(minutes=5),
    # 'queue': 'bash_queue',
    # 'pool': 'backfill',
    # 'priority_weight': 10,
    # 'end_date': datetime(2016, 1, 1),
}

dag = DAG(
    'tutorial', default_args=default_args, schedule_interval=timedelta(1))

# t1, t2 and t3 are examples of tasks created by instantiating operators
t1 = BashOperator(
    task_id='print_date',
    bash_command='date',
    dag=dag)

t2 = BashOperator(
    task_id='sleep',
    bash_command='sleep 5',
    retries=3,
    dag=dag)

templated_command = """
    {% for i in range(5) %}
        echo "{{ ds }}"
        echo "{{ macros.ds_add(ds, 7)}}"
        echo "{{ params.my_param }}"
    {% endfor %}
"""

t3 = BashOperator(
    task_id='templated',
    bash_command=templated_command,
    params={'my_param': 'Parameter I passed in'},
    dag=dag)

t2.set_upstream(t1)
t3.set_upstream(t1)

The tutorial.py is under ~/airflow/dags. By running airflow list_dags I can see the tutorial at the end of list. However, when I run airflow test tutorial print_date 2018-09-04, it only prints:

[2018-09-04 22:14:43,096] {__init__.py:51} INFO - Using executor SequentialExecutor
[2018-09-04 22:14:43,199] {models.py:258} INFO - Filling up the DagBag from /Users/chenyuanfei/airflow/dags

And nothing else. I'm using apche-airflow 1.10 on OSX. How can I correctly run the script?

is the dag switched on? it may not run if it's switched off (by default) in the web UI — user3582076
This shouldn't impact running tests. Switching on is when you want to schedule it. But you can run test or backfill from the command line. What if you try to run backfill instead of test? — Tomasz Krol
@Tomasz Krol It outputs INFOs such as [2018-09-05 11:11:25,025] {base_executor.py:56} INFO - Adding to queue: airflow run example_hello_world_dag date_task 2015-06-05T00:00:00+00:00 --local -sd DAGS_FOLDER/tutorial.py --cfg_path /var/folders/mg/kknt930j1_n4ppyn2ydm4z2h0000gn/T/tmpZ_kdgv, but still nothing I expected was printed. I'm doubting if it's because I have both airflow1.8 and apache-airflow1.10 on my mac. I uninstalled both and reinstalled airflow1.8, and this time it works — WTIFS

WTIFS WTIFS · Accepted Answer · 2018-09-27T10:10:03

I'm doubting if it's because I have both airflow1.8 and apache-airflow1.10 on my mac. I uninstalled both and reinstalled airflow1.8, and this time it works

BashOperator doen't run bash command apache airflow

1 Answers