runtimes

Background compute runtimes for Bitfount.

Each sub-package in this directory defines a Prefect flow that is triggered by an external signal (e.g. dataset creation) and runs a background compute job independently of the main pod process.

Available runtimes

bitfount.runtimes.file_metadata Triggered when a dataset (pod) is created. Scans the datasource's files (or DB connection) and writes one FileMetadata per file into the local SQLite metadata cache.

Module

Submodules

• bitfount.runtimes.bookkeeping - Shared run scaffolding for the background metadata flows.
• bitfount.runtimes.dedup - Prefect-native liveness and dedup for background work.
• bitfount.runtimes.deployments - Deployment identity and readiness for the background metadata runtimes.
• bitfount.runtimes.discovery - Reading recoverable flow runs off a Prefect server.
• bitfount.runtimes.file_metadata - file_metadata runtime package.
• bitfount.runtimes.prefect_bootstrap - Idempotent Prefect bootstrapping for the metadata runtimes.
• bitfount.runtimes.recovery - Recovery for metadata-runtime flow runs whose process is gone.
• bitfount.runtimes.scan_metadata - Runtime for the scan_metadata cache table.

---