Skip to main content

result_base

Shared base for step results backed by a cache partition.

Cache-backed steps (those declaring an orm_model) return a result that carries a row count plus the CacheAccessor backing <step>.cache. The shape is identical across steps, so it lives here once: subclasses add only a docstring. The accessor is attached after construction by the producing task (or, cross-phase, by BackgroundResultsContext).

Module​

Functions​

result_output_fields​

def result_output_fields(result_cls: type[pydantic.main.BaseModel]) ‑> frozenset[str]:

Return the output names a DAG input ref may target on result_cls.

A step result exposes an output either as a declared pydantic field (ScanFilterResult.filenames) or as a property reading through to the cache (CacheBackedResult.cache / .records). Both are legitimate FromRef/BackgroundRef targets. Pydantic's own inherited properties (model_extra, model_fields_set) and plain methods are not outputs, even though getattr would find them.

Arguments

  • result_cls: A step's registered Result class.

Returns The set of names a ref's output_field may name.

Classes​

CacheBackedResult​

class CacheBackedResult(**data: Any):

Base for step results whose rows live in a cache partition.

Attributes

  • records_stored: How many rows this result stands for — those the invocation persisted, or, where it persisted none, the rows already in the partition it stands for. NOT a measure of work done by this run, and not safe to use as one: the producers deliberately differ. A step that persisted rows reports what it wrote; a step that found everything current reports accessor.count(); the cross-phase reconstruction in BackgroundResultsContext does the same, having never written anything itself; and model_inference's all-cached skip reports the count of selected files. Each subclass's docstring states which of these its own paths produce. Anything that needs rows-written-this-run must be given its own field rather than reinterpreting this one.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Subclasses​

Variables​

  • static model_config
  • static records_stored : int
  • cache : CacheAccessor | None - Return the CacheAccessor for reading this result in a DAG pipeline.

  • records : pandas.core.frame.DataFrame - Retrieve this result's rows as a DataFrame.

    The DataFrame is fetched from the cache on each access — it is not stored in the result object, allowing lazy access to large datasets without materialising them in memory until needed.

    Returns: A pandas.DataFrame with one row per cached record.

    Raises: RuntimeError: If no accessor is attached (e.g. the result was serialised across a Prefect task boundary). In a DAG pipeline use <step_name>.cache instead.

Static methods​

from_accessor​

def from_accessor(    accessor: CacheAccessor, *, records_stored: int | None = None,) ‑> Self:

Build a result carrying accessor, in one call rather than two.

_accessor is a PrivateAttr, so it cannot be passed to the constructor and every producer would otherwise construct the result and then reach in to attach the accessor. That two-step is the shape this replaces.

Arguments

  • accessor: The CacheAccessor backing <step>.cache, scoped to the partition this result stands for.
  • records_stored: Rows this result stands for. Defaults to accessor.count() — the whole partition — which is what a step that persisted nothing this run reports. A step that wrote rows passes what it wrote; see the field's docstring for why the two are not interchangeable.

Returns An instance of the calling subclass, with the accessor attached.

Methods​

model_post_init​

def model_post_init(self: BaseModel, context: Any, /) ‑> None:

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that's what pydantic-core passes when calling it.

Arguments

  • self: The BaseModel instance.
  • context: The context.