Geek Answers Handbook

Airflow: crontab time change for DAG in Airflow

I have a DAG that runs every day at 3:00, it went fine over the past few weeks.

I updated the launch date at 7:00, but apparently the last 2 days it did not start. I see tasks for these two days with the status "running" (green), but no command is launched.

Does something more need to be done to change DAG runtimes?

I know that in the past, one way to solve this problem was to clear the tasks for this DAG in the meta database and update start_date, but I would rather not do it again.

Anyone have a suggestion?

+8
source share
3 answers

To schedule an error, Airflow simply looks for the latest execution date and summarizes the schedule interval . If this time is up, it will launch a dag. You cannot just update the start date. An easy way to do this is to edit the start date and schedule interval , rename your dag (e.g. xxxx_v2.py) and redeploy it.

+9
source

An alternative solution for renaming a DAG is to edit the execution_date all instances of the previous task and execute the DAG DAG in the database. Tables for change: task_instance and dag_run respectively.

One of the drawbacks of this approach is that you will lose the ability to view completed task logs through a web server.

+4
source

David,
1. You can also remove a shortcut through the Experimental REST API. DAG removal
2. Change the desired start_date start.
3. And add the same DAG back.

0
source

More articles:


All Articles