The CodeDeploy got failed when the local airflow.cfg is packaged in zip file

1
votes

I tried to deploy airflow DAGs to AWS EC2. I followed the instruction on aws-airflow-stack. The airflow runs well on EC2. In EC2 Instances, there are two airflow.cfg files, they are located at:

/airflow/airflow.cfg
/home/ec2-user/airflow/airflow.cfg

In folder '/airflow', the Dags data existed there. In folder '/home/ec2-user/airflow', it just has airflow.cfg, airflow.db, logs and unittests.cfg files. Both with the default settings.

Now I need to add some users on Postgers database, I put the [webserver] authenticate = true and rbac = true in my local airflow.cfg, and deploy this config file with CodeDeploy, the deployment got failed. Check the log, In install event. the message is " The deployment failed because a specified file already exists at this location: /airflow/airflow.cfg". How do I address this issue? how to deploy a local airflow config file to EC2? Any suggestion is welcome.

1
airflowaws-code-deploy

1 Answers

1
votes

Can you also provide the command you used to trigger CodeDeploy?

The command in the example Makefile specifies that files already existing should be overwritten:

aws deploy create-deployment \
    --application-name $(APPLICATION) \
    --deployment-group-name $(DEPLOYMENT_GROUP) \
    --s3-location bucket=$(DEPLOYMENTS_BUCKET),bundleType=tgz,key=$(PACKAGE) \
    --deployment-config-name CodeDeployDefault.AllAtOnce \
    --file-exists-behavior OVERWRITE

Special attention to the --file-exists-behavior OVERWRITE part. Using the Makefile to deploy will ensure your airflow.cfg file will replace the existing one.