Introduction to Prefect and Workflow Orchestration

Mar 2, 2022 · St. Louis, United States of America

Workflow management systems are used for scheduling and monitoring data pipelines. This includes managing task dependencies, retrying failed tasks, and sending notifications to users. This talk will show data engineers and scientists how to orchestrate their data workflows with Prefect, an open-source modern workflow management system designed with Dask natively built-in. After this talk, attendees should understand the basics of workflow orchestration and how to get started implementing it for their use cases

In an interactive demo, we'll go over Prefect basic concepts such as Flows, Tasks, and Parameters. We'll then move on to more advanced topics such as parallelism and conditional logic, which let us dynamically create Tasks inside a Flow. During the demo, we will deploy a Flow locally, and then show how seamless it is to port the Flow to a Dask cluster on the cloud.

Agenda:
5:30-5:40 - Introductions
5:40-6:40 - Presentation
6:40-7:00 - Questions

Bio:

Kevin Kho is an Open Source Community Engineer at Prefect, an open-source workflow orchestration management system. Previously, he was a data scientist at Paylocity, where he worked on adding machine learning features to their Human Capital Management (HCM) Suite. Outside of work, he is a contributor for Fugue, an open-source abstraction layer for distributed computing. He also organizes the Orlando Machine Learning and Data Science Meetup.

Event organizers

Are you organizing Introduction to Prefect and Workflow Orchestration?

Claim the event and start manage its content.

I am the organizer
Social
Topics
Rating

based on 0 reviews