Files
oam/knowledge base/prefect.md
2025-03-25 20:59:23 +01:00

1.4 KiB

Prefect

Workflow orchestration framework for building data pipelines in Python.

Workflow activity is tracked and can be monitored from a Prefect (self-hosted or cloud-managed).

Allows building and scheduling workflows in pure Python, to then run them anywhere.
Designed to handle retries, dependencies, branching logic, dynamic workflows, modern infrastructure, and data pipelines' complexity.
Handles automatic state tracking, failure handling, real-time monitoring, and more.

  1. TL;DR
  2. Further readings
    1. Sources

TL;DR

Setup
# Install.
docker pull 'prefecthq/prefect:3-latest'
pip install --upgrade 'prefect'

# Check installation.
prefect version
Usage
# Start.
prefect server start
docker run -d -p '4200:4200' 'prefecthq/prefect:3-latest' -- prefect server start --host '0.0.0.0'

Further readings

Sources