GoDataDriven open source contribution: April 2017 edition
Third edition already: time flies! Three is a magic number so we went, without any planning in typical GoDataDriven style, with an Airflow special!
- Henk added Azure Blob Storage with PR 2216, refactored tests to that Travis would pick them up with PR 2234, and finally captured invalid arguments for Sqoop in PR 2252;
- Alexander added Kerberos support when using Python 3 in PR 2158 (this is kind of a big deal!);
- Niels added support to allow for custom filters in Jinja2 templates in PR 2258;
- Yours truly fixed the scripts execution in the SparkSql hook with PR 2259.
The list of what we've done does not end here though! Robert, after being gently pushed, made his very first contribution to Homebrew. As he started playing with the ELK stack on his Mac he noted that Logstash did not have a service associated with it. He fixed it with PR 12144 that, after some internal discussion, got merged.
- In PR 4210 he updated the pom to Parquet 1.8.2 (whose release notes are MIA: if you find them, let him know!);
- In PR 4233 he removed a dependency, as less is more, especially with dependencies!
Last, but really not least, Kris open sourced smokey, a Hadoop smoke testing framework. He put quite a bit of effort into it, so check it out! Also worth noting that our blog is, at least internally, read, so he borrowed the Python project structure that Henk presented in his post.
That's it for this edition! As always, if you have any comments, remarks, or compliments, we'd love to hear them!
Follow us for more of this
GoDataDriven open source contribution: December 2018 edition
January 14, 2019
Using the Airflow Experimental Rest API to trigger a DAG
January 12, 2019
Apache Airflow graduation as Apache Top-Level
January 08, 2019
Data Survey 2018/2019 - Data 50
January 07, 2019
Use a SSH-key to access your cloud resources with socks-proxy
December 31, 2018
Looking Back at our Deep Learning Frenzy
December 28, 2018