4.58 out of 5
4.58
623 reviews on Udemy

Apache Airflow: The Hands-On Guide

Master Apache Airflow from A to Z. Hands-on videos on Airflow with AWS, Kubernetes, Docker and more
Instructor:
Marc Lamberti
4,736 students enrolled
English
Coding Production Grade Data pipelines by Mastering Airflow through Hands-on Examples
How to Follow Best Practices with Apache Airflow
How to Scale Airflow with the Local, Celery and Kubernetes Wxecutors
How to Set Up Monitoring with Elasticsearch and Grafana
How to Secure Airflow with authentication, crypto and the RBAC UI
Core and Advanced Concepts with Pros and Limitations
Mastering DAGs with timezones, unit testing, backfill and catchup
Organising the DAG folder and keep things clean

Apache Airflow is a platform created by community to programmatically author, schedule and monitor workflows.

It is scalable, dynamic, extensible and modulable.

Without any doubts, mastering Airflow is becoming a must-have and an attractive skill for anyone working with data.

What you will learn in the course:

  • Fundamentals of Airflow are explained such as what is Airflow, how the scheduler and the web server works

  • The Forex Data Pipeline project is incredible way to discover many operators in Airflow and deal with Slack, Spark, Hadoop and more

  • Mastering your DAGs is a top priority and you will be able to play with timezones, unit testing your DAGs, how to structure your DAG folder and much more

  • Scaling Airflow through different executors such as the Local Executor, the Celery Executor and the Kubernetes Executor will be explained in details. You will discover how to specialise your workers, how to add new workers, what happens when a node crashes.

  • A Kubernetes cluster of 3 nodes will be set up with Rancher, Airflow and the Kubernetes Executor in local to run your data pipelines.

  • Advanced concepts will be shown through practical examples such as templatating your DAGs, how to make your DAG dependent of another, what are Subdags and deadlocks, and more.

  • You will set up a Kubernetes cluster in the cloud with AWS EKS and Rancher  in order to use Airflow along with the Kubernetes Executor

  • Monitoring Airflow is extremely important! That’s why you will know how to do it with Elasticsearch and Grafana.

  • Security will be also addressed in order to make your Airflow instance compliant with your company. Specifying roles and permissions for your users with RBAC, Prevent from accessing the Airflow UI with authentication and password,  data encryption and more.

In addition:

  • Many practical exercises are given along the course so that you will have occasions to apply what you learn.

  • Best practices are stated when needed to give you the best ways of using Airflow

  • Quiz are available to assess your comprehension at the end of each section.

  • Answering fast your questions is my top-priority and I will do my best for you.

I put a lot of effort in order to give you the best content and I hope you will enjoy it as much as I enjoyed doing it.

At the end of the course you will more confident than ever to use Airflow

Wish you a great success!

Marc Lamberti

You can view and review the lecture materials indefinitely, like an on-demand channel.
Definitely! If you have an internet connection, courses on Udemy are available on any device at any time. If you don't have an internet connection, some instructors also let their students download course lectures. That's up to the instructor though, so make sure you get on their good side!
4.6
4.6 out of 5
623 Ratings

Detailed Rating

Stars 5
387
Stars 4
183
Stars 3
35
Stars 2
9
Stars 1
7
a55f0d0e4d18c9a06d60f9474b2d8851
30-Day Money-Back Guarantee

Includes

13 hours on-demand video
4 articles
Full lifetime access
Access on mobile and TV
Certificate of Completion