Skip to main content

filters

Trial inclusion filters for steps.

This package is the canonical home for ColumnFilter and MethodFilter used across multiple steps for eligibility criteria filtering.

Sub-modules​

column ColumnFilter — filters a DataFrame on a single named column. method MethodFilter — filters a DataFrame using an arbitrary callable. _telemetry Private helper for emitting TrialFilterSummaryEvent telemetry.

Module​

Submodules​

Functions​

normalize_column_name​

def normalize_column_name(name: str) ‑> str:

Normalize a column name for case/space-insensitive comparison.

Lower-cases and strips spaces so "totalgaarea" matches "Total GA Area". The single source of truth for how a column name is resolved — matched_column (and grain resolution in criteria_matching.functions) call this so they cannot drift apart.

Arguments

  • name: The raw column name.

Returns The normalized name.

Classes​

ColumnFilter​

class ColumnFilter(    column: str,    operator: str,    value: str | int | float | list[str | int | float],    how: PartialMatchingType = 'all',    config_field: str | None = None,    output_field: str | None = None,    grain: CriterionGrain = scan,    family: CriterionFamily | None = None,    tree_node_id: str | None = None,):

Dataclass for basic single column filtering.

Arguments

  • column: The column name on which the filter will be applied.
  • operator: The operator for the filtering operation.
  • value: The value for the filter.
  • how: Partial matching type ("any" or "all").
  • config_field: Originating CriteriaMatchConfig field name, or None if this filter was not built from a config (e.g. constructed directly in a test).
  • output_field: Logical field name this filter's outcome is reported under (typically the filter's column), or None if unset.
  • grain: Whether this filter is evaluated per scan or per patient. Defaults to CriterionGrain.SCAN. This is the grain (cardinality) axis, not provenance — a criterion's data source is resolved from its column in build_criterion_outcomes, not set here.
  • family: The semantic criterion family this filter belongs to (see CriterionFamily), declared by the build site in build_eligibility_filters. Pod-internal: a task config authoring a ColumnFilter by hand leaves it unset, which is why it is optional rather than required. None therefore means "not built by build_eligibility_filters", and the served criterion reports no family.
  • tree_node_id: The criteria_tree leaf id this filter was built from, or None for a filter not built from a tree leaf. Stamped onto the resulting CriterionOutcome by build_criterion_outcomes.

A ColumnFilter is one declarative, self-describing scalar comparison: a single column compared against a scalar value (or, for in/not in, a value-set) with an operator, evaluated vectorized over the column. Use it whenever the criterion is a single-column scalar comparison. For anything that is not — multi-column rules, list/collection-valued cells, glob/regex matching — use a MethodFilter instead. A produced CriterionOutcome from a ColumnFilter carries kind="column".

Variables​

  • static column : str
  • static config_field : str | None
  • static how : Literal['any', 'all']
  • static operator : str
  • static output_field : str | None
  • static tree_node_id : str | None
  • static value : str | int | float | list[str | int | float]
  • identifier : str - Identifying string for this filter to use in logs.

Methods​

apply_filter​

def apply_filter(    self, df: pd.DataFrame, rename_columns: typing.Mapping[str, str] | None = None,) ‑> pandas.core.frame.DataFrame:

Apply self filter to a dataframe.

evaluate_row​

def evaluate_row(    self, row: pd.Series, matched: str | None,) ‑> CriterionOutcome:

Evaluate this filter against one row, returning a structured outcome.

Self-contained: computes pass/fail directly from the operator (mirroring apply_filter's op(value) & notnull), so it does not depend on apply_filter having written any column. An absent target column or a NaN value yields unknown; absent_column distinguishes the two so the regenerated "not found" reason can render the raw column name (absent) rather than its display rename (present but NaN), matching apply_filter.

Arguments

  • row: The df row.
  • matched: The resolved df column (from matched_column), or None.

Returns The CriterionOutcome for this filter on this row.

matched_column​

def matched_column(self, df: pd.DataFrame) ‑> str | None:

Resolve the actual df column this filter targets, or None.

Matching is case- and space-insensitive (e.g. "totalgaarea" matches "Total GA Area"), mirroring the resolution apply_filter performs.

Arguments

  • df: The DataFrame whose columns are searched.

Returns The matching column name, or None if no column matches.

result_column​

def result_column(self, matched: str) ‑> str:

The per-filter boolean column name apply_filter writes.

Arguments

  • matched: The resolved df column name (from matched_column).

Returns The column name holding this filter's per-row pass/fail bool.

MethodFilter​

class MethodFilter(    method: typing.Callable[[pd.Series], MethodOutcome | tuple[bool | None, str | None]],    required_columns: set[str],    filter_name: str,    filter_failed_message: str,    config_field: str | None = None,    output_field: str | None = None,    grain: CriterionGrain = scan,    provenance_column: str | None = None,    operator: str | None = None,    operand: typing.Any = None,    family: CriterionFamily | None = None,    tree_node_id: str | None = None,):

Dataclass for filtering using a python method.

Arguments

  • method: Filter method returning a MethodOutcome, or the bare (eligible, context) tuple that is its first two fields. A None eligible signals unknown (e.g. the patient's data for this filter is absent) and maps to CriterionState.UNKNOWN. A filter with structured detail to report (today, the code-list filters) returns the third field too; everything else keeps returning a 2-tuple. A filter reading more than one source column returns the fourth field, naming the column this row's verdict came from, so build_criterion_outcomes can stamp provenance per row rather than per filter (today, only the observation filter).
  • required_columns: Columns required in the df.
  • filter_name: Name of filter.
  • filter_failed_message: Message to explain patient ineligibility.
  • config_field: Originating CriteriaMatchConfig field name, or None if this filter was not built from a config (e.g. constructed directly in a test).
  • output_field: Logical field name this filter's outcome is reported under (typically the filter's filter_name), or None if unset.
  • grain: Whether this filter is evaluated per scan or per patient. Defaults to CriterionGrain.SCAN. This is the grain (cardinality) axis, not provenance — a criterion's data source is resolved from its column in build_criterion_outcomes, not set here.
  • provenance_column: The single source column this filter's provenance should be read from. Leave None for a single-column filter — its sole required_columns entry is used automatically. A filter with more than one required column has no implicit source; set this to the column that determines its provenance (otherwise it resolves to unknown).
  • operator: The evaluation operator, or None when the filter's predicate is not a single named comparison (the typed compound filters) or its operand is deliberately not recorded (the code lists).
  • operand: The configured value the operator compared against, or None. Set only by the generic column_criteria paths, which know their own operator; see CriterionOutcome.operand for why it is not threshold.
  • family: The semantic criterion family this filter belongs to (see CriterionFamily), declared by the build site. None for a filter constructed directly rather than by build_eligibility_filters.
  • tree_node_id: The criteria_tree leaf id this filter was built from, or None for a filter not built from a tree leaf. Stamped onto the resulting CriterionOutcome by build_criterion_outcomes.

A MethodFilter is an arbitrary per-row predicate for criteria a single declarative scalar comparison cannot express: multi-column rules, list/collection-valued cells (e.g. EHR code lists), and glob/regex matching. It runs per row rather than vectorized. Prefer a ColumnFilter for a plain single-column scalar comparison. A produced CriterionOutcome from a MethodFilter carries kind="method".

Variables​

  • static config_field : str | None
  • static filter_failed_message : str
  • static filter_name : str
  • static operand : Any
  • static operator : str | None
  • static output_field : str | None
  • static provenance_column : str | None
  • static required_columns : set[str]
  • static tree_node_id : str | None
  • identifier : str - Identifying string for this filter to use in logs.

Methods​

apply_filter​

def apply_filter(    self, df: pd.DataFrame, **kwargs: typing.Any,) ‑> pandas.core.frame.DataFrame:

Apply self filter to a dataframe.

evaluate_row​

def evaluate_row(    self, row: pd.Series, matched: str | None = None,) ‑> CriterionOutcome:

Evaluate this method filter against one row, returning a structured outcome.

Self-contained: runs self.method once. A missing required column or a non-bool result yields unknown (never pass), matching the build_patient_level_evidence rule. On failure, failed_message is regenerated to match apply_filter: the static message for a clean failure, "missing values for ... filter" when a required column is present but NaN, or "unable to determine ... filter" when a required column is absent altogether.

Arguments

  • row: The df row.
  • matched: Unused; accepted for signature symmetry with ColumnFilter.

Returns The CriterionOutcome for this filter on this row.