Exploring the Power of Airflow 3 at Astronomer with Amogh Desai

The Data Flowcast: Mastering Apache Airflow ® for Data Engineering and AI

What does it take to go from fixing a broken link to becoming a committer for one of the world’s leading open-source projects?

Amogh Desai, Senior Software Engineer at Astronomer, takes us through his journey with Apache Airflow. From small contributions to building meaningful connections in the open-source community, Amogh’s story provides actionable insights for anyone on the cusp of their open-source journey.

Key Takeaways:

(02:09) Building data engineering platforms at Cloudera with Kubernetes.

(04:00) Brainstorming led to contributing to Apache Airflow.

(05:17) Starting small with link fixes, progressing to Breeze development.

(07:00) Becoming a committer for Apache Airflow in September 2023.

(09:51) The steep learning curve for contributing to Airflow.

(16:30) Using GitHub’s “good-first-issue” label to get started.

(18:15) Setting up a development environment with Breeze.

(22:00) Open-source contributions enhance your resume and career.

(24:51) Amogh’s advice: Start small and stay consistent.

(28:12) Engage with the community via Slack, email lists and meetups.

Resources Mentioned:

Amogh Desai -

https://www.linkedin.com/in/amogh-desai-385141157/?originalSubdomain=in%20%20https://www.linkedin.com/company/astronomer/

Astronomer -

https://www.linkedin.com/company/astronomer/

Apache Airflow GitHub Repository -

https://github.com/apache/airflow

Contributors Quick Guide -

https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst

Breeze Development Tool -

https://github.com/apache/airflow/tree/main/dev/breeze

Thanks for listening to “The Data Flowcast: Mastering Airflow for Data Engineering & AI.” If you enjoyed this episode, please leave a 5-star review to help get the word out about the show. And be sure to subscribe so you never miss any of the insightful conversations.

#AI #Automation #Airflow #MachineLearning