Skip to main content

parser

Parser: converts a v9 FlowSpec into a BackgroundDAG.

The parser is intentionally side-effect-free. It reads the FlowSpec produced by the existing schema deserialiser (flows/schema.py) and produces a BackgroundDAG ready for validation and execution.

Responsibilities​

  • Parse raw YAML input reference strings into typed FromRef / ContextRef objects.
  • Map the config: block (keyed by step name) to the step's Pydantic config class (looked up by task name + version from the step registry).
  • Merge the shared config.runtime block into every step's config dict before validation (step-specific keys take precedence).
  • Warn — but do not abort — when the YAML config contains fields that are unknown to the Pydantic class.
  • Raise LegacyTaskYAMLError early when given a non-v9 FlowSpec.

What the parser does NOT do​

  • Registry existence checks — that is validator.py's job.
  • I/O of any kind.

Module​

Functions​

parse_background_dag​

def parse_background_dag(flow_spec: FlowSpec) ‑> BackgroundDAG:

Parse the background portion of a v9 FlowSpec into a BackgroundDAG.

Iterates over flow_spec.worker.background.steps, parses each step's input references, instantiates Pydantic configs from the config: block, and returns a BackgroundDAG populated with DAGStep instances.

Arguments

  • flow_spec: A fully deserialised v9 FlowSpec (from bitfount.flows.schema).

Returns A BackgroundDAG ready for validation (validator.py) and execution (executor.py).

Raises

  • LegacyTaskYAMLError: If flow_spec does not look like a v9 flow (missing federation, roles, or worker).
  • ValueError: If an input reference is malformed, or a config cannot be validated against its Pydantic class.

parse_interactive_dag​

def parse_interactive_dag(    flow_spec: FlowSpec,) ‑> InteractiveDAG:

Parse the interactive portion of a v9 FlowSpec into an InteractiveDAG.

Iterates over flow_spec.worker.interactive.steps. References to background steps (step.field where the step is declared in flow_spec.worker.background) become BackgroundRef inputs resolved from cache at runtime; references to other interactive steps remain FromRef; $provider.field remain ContextRef.

The parsed background steps (with their configs) are carried on the returned InteractiveDAG so the run-context builder can construct the BackgroundResultsContext and the validator can check BackgroundRef targets.

Arguments

  • flow_spec: A fully deserialised v9 FlowSpec.

Returns An InteractiveDAG ready for validation and execution.

Raises

  • LegacyTaskYAMLError: If flow_spec does not look like a v9 flow.
  • ValueError: If an input reference is malformed, or a config cannot be validated against its Pydantic class.

Classes​

LegacyTaskYAMLError​

class LegacyTaskYAMLError(*args, **kwargs):

Raised when a non-v9 FlowSpec is passed to the background DAG parser.

v9 flows are identified by the presence of federation, roles, and worker fields. Older task YAML formats (v8 and below) use a flat ModellerConfig structure and are not supported by this parser.