Migrating your Hadoop workloads to the Cloud
GoDataDriven announces Data Council NL community
GoDataDriven Open Source Contribution for February 2019, the first Open Source Initiatives edition
Open Sourcing Airflow Local Development
A Practical Guide to Using Setup.py
Docker Hub Tips and Tricks
How to style transfer your own images
It's time to trust your predictions
Testing and debugging Apache Airflow
One of the questions I get asked the most about Apache Airflow is how to shorten the development cycle of pushing code, deploying, and manually triggering a DAG for verification to something that is locally testable without running on a live system. In this blog post I provide several pointers to testing and debugging Apache Airflow on your local machine.
The Zen of Python and Apache Airflow
Apache Airflow is a Python framework for programmatically creating workflows in DAGs. This allows for concise and flexible scripts but can also be the downside of Airflow; since it's Python code there are infinite ways to define your pipelines. The Zen of Python is a list of 19 Python design principles and in this blog post I point out some of these principles on four Airflow examples.