Skip to main content

v1

Scan-eligibility step v1.

Module​

Submodules​

Functions​

task_fn​

def task_fn(    evaluations: list[CriteriaEvaluation],    cache: CacheProtocol,    task_hash: str,    project_id: str,    config: ScanEligibilityConfig | None = None,    run_id: str | None = None,) ‑> ScanEligibilityResult:

Compute + persist per-scan (scan-level) eligibility.

Arguments

  • evaluations: Per-scan CriteriaEvaluations from criteria_matching (new scans only — upstream scopes them via get_unprocessed_files).
  • cache: The pod background cache to write to.
  • task_hash: Partition key; a config change lands in a fresh partition.
  • project_id: The trial (project) ID (a runtime param).
  • config: Optional step config (trial display name).
  • run_id: Optional provenance run ID.

Returns ScanEligibilityResult with the number of scan_eligibility rows written and a cache accessor scoped to this task_hash.

Raises

  • ValueError: If project_id is not supplied.

Classes​

Config​

class Config(**data: Any):

Config for the scan_eligibility step.

Arguments

  • trial_name: Optional display name for the trial, stored for the UI.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Variables​

  • static model_config
  • static trial_name : str | None

Result​

class Result(**data: Any):

Result of a scan_eligibility task.

records_stored reports how many scan_eligibility rows were persisted; cache exposes the CacheAccessor over them for the downstream reduce step.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Ancestors​

Variables​

  • static model_config
  • cache : CacheAccessor | None - Return the CacheAccessor for reading this result in a DAG pipeline.

  • records : pandas.core.frame.DataFrame - Retrieve this result's rows as a DataFrame.

    The DataFrame is fetched from the cache on each access — it is not stored in the result object, allowing lazy access to large datasets without materialising them in memory until needed.

    Returns: A pandas.DataFrame with one row per cached record.

    Raises: RuntimeError: If no accessor is attached (e.g. the result was serialised across a Prefect task boundary). In a DAG pipeline use <step_name>.cache instead.

Static methods​

from_accessor​

def from_accessor(    accessor: CacheAccessor, *, records_stored: int | None = None,) ‑> Self:

Inherited from:

CacheBackedResult.from_accessor :

Build a result carrying accessor, in one call rather than two.

_accessor is a PrivateAttr, so it cannot be passed to the constructor and every producer would otherwise construct the result and then reach in to attach the accessor. That two-step is the shape this replaces.

Arguments

  • accessor: The CacheAccessor backing <step>.cache, scoped to the partition this result stands for.
  • records_stored: Rows this result stands for. Defaults to accessor.count() — the whole partition — which is what a step that persisted nothing this run reports. A step that wrote rows passes what it wrote; see the field's docstring for why the two are not interchangeable.

Returns An instance of the calling subclass, with the accessor attached.

Methods​

model_post_init​

def model_post_init(self: BaseModel, context: Any, /) ‑> None:

Inherited from:

CacheBackedResult.model_post_init :

This function is meant to behave like a BaseModel method to initialise private attributes.

It takes context as an argument since that's what pydantic-core passes when calling it.

Arguments

  • self: The BaseModel instance.
  • context: The context.