The Machine Learning Podcast
Detailed and technical explorations of machine learning and artificial intelligence with the researchers, engineers, and entrepreneurs who are shaping the industry
We found 1 episode of The Machine Learning Podcast with the tag “data orchestration”.
Build More Reliable Machine Learning Systems With The Dagster Orchestration Engine
December 1st, 2022 | 45 mins 43 secs
data orchestration, mlops
Building a machine learning model one time can be done in an ad-hoc manner, but if you ever want to update it and serve it in production you need a way of repeating a complex sequence of operations. Dagster is an orchestration engine that understands the data that it is manipulating so that you can move beyond coarse task-based representations of your dependencies. In this episode Sandy Ryza explains how his background in machine learning has informed his work on the Dagster project and the foundational principles that it is built on to allow for collaboration across data engineering and machine learning concerns.