Sunday, 2 May 2021

Return value from one Airflow DAG into another one

My DAG (let's call it DAG_A) starting another DAG (DAG_B) using trigger_dagrun operator. DAG_B's tasks use XCOM and I would like to obtain XCOM value from one of the tasks of DAG_B's run (exactly the one I've started) upon completion.

Use of XCOM is not a hard requirement - basically any (reasonable) mechanism that Airflow itself provides would work. I can change DAG_B if needed.

Can't find any examples of such cases, so appreciate the help.

Plan B would be to make DAG_B save XCOM values into some persistent storage like DB or file together with some run id, and DAG_A will take it from there. But I would like to avoid such complications if some built-in mechanisms available.



from Return value from one Airflow DAG into another one

No comments:

Post a Comment