Member-only story
9 technologies that changed data engineering forever
Without them, we would live in a very different world
To sustain my writing, I’ve put this article behind a paywall. If you have a Medium membership, I’d really appreciate it. If you prefer to read it for FREE, please visit here and subscribe to my newsletter. Either way, you’ll help keep my work going.
Intro
Data engineering is the development, implementation, and maintenance of systems and processes that take in raw data and produce high-quality, consistent information that supports downstream use cases, such as analysis and machine learning
“Systems” and “processes“ are a lot of things. How do you store the data? How do you process the data? How do you “glue“ transform steps together?
Thankfully, we don’t develop them from scratch most of the time. There are technologies, tools, or frameworks. Some of them not only help, but change the field.
In this article, I will discuss 9 technologies (in my opinion) that have changed data engineering forever.
