Skip to main content

filtering_algorithm

Algorithm for filtering data records based on configurable strategies.

Classes​

AgeRangeFilterArgs​

class AgeRangeFilterArgs(*args, **kwargs):

Arguments for AGE_RANGE filter strategy.

This filtering strategy keeps only records within a specified age range in a given column.

Variables​

  • static birth_date_column : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static max_age : int
  • static min_age : int
  • static remote_modeller : bool

ContinuousScanHistoryFilterArgs​

class ContinuousScanHistoryFilterArgs(*args, **kwargs):

Arguments for CONTINUOUS_SCAN_HISTORY filter strategy.

Keeps patients that have at least one contiguous chain of scans where every gap between consecutive scans (sorted by date) is at most max_continuous_history_gap_months months, and the total span of that chain (oldest to newest scan in the chain) is at least min_continuous_history_total_years years.

All records for qualifying patients are retained; the strategy makes a keep/discard decision at the patient (ID) level, not per scan.

Recency is optional: when max_months_since_last_scan is provided, the qualifying chain must also end within that many months of today, so the continuous history is anchored to the present. When omitted, recency is not checked (a qualifying chain anywhere in history is sufficient) — pair with scan_in_period if a separate recency gate is preferred.

Arguments

  • date_column: Column containing scan/acquisition dates.
  • id_column: Column(s) to group patients by.
  • min_continuous_history_total_years: Minimum required span (in years) from the first to the last scan in the qualifying chain.
  • max_continuous_history_gap_months: Maximum allowed gap (in months) between consecutive scans within the qualifying chain.
  • max_months_since_last_scan: Optional recency window (in months). When set, the qualifying chain's most recent scan must fall within this many months of today.

Variables​

  • static date_column : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static max_continuous_history_gap_months : int
  • static max_months_since_last_scan : int
  • static min_continuous_history_total_years : int
  • static remote_modeller : bool

FilterStrategy​

class FilterStrategy(*args, **kwds):

Enumeration of available filtering strategies.

Ancestors​

Variables​

  • static AGE_RANGE
  • static CONTINUOUS_SCAN_HISTORY
  • static FREQUENCY
  • static LATEST
  • static NUMERIC_THRESHOLD
  • static PATIENT_ID
  • static PATIENT_ID_WITH_PREVIOUS_RUNS
  • static REQUIRED_COLUMN_PRESENT
  • static SCAN_FREQUENCY
  • static SCAN_IN_PERIOD
  • static SERIES_DESCRIPTION_LATEST

FilterStrategyClass​

class FilterStrategyClass(*args, **kwds):

Enumeration map of filter strategies to TypedDict and classes.

Ancestors​

Variables​

  • static AGE_RANGE
  • static CONTINUOUS_SCAN_HISTORY
  • static FREQUENCY
  • static LATEST
  • static NUMERIC_THRESHOLD
  • static PATIENT_ID
  • static PATIENT_ID_WITH_PREVIOUS_RUNS
  • static REQUIRED_COLUMN_PRESENT
  • static SCAN_FREQUENCY
  • static SCAN_IN_PERIOD
  • static SERIES_DESCRIPTION_LATEST

FrequencyFilterArgs​

class FrequencyFilterArgs(*args, **kwargs):

Arguments for FREQUENCY filter strategy.

This filtering strategy keeps only records with a specified frequency of ID occurrence.

Variables​

  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static max_frequency : int
  • static min_frequency : int
  • static remote_modeller : bool

LatestFilterArgs​

class LatestFilterArgs(*args, **kwargs):

Arguments for LATEST filter strategy.

This filtering strategy keeps only the latest records per ID.

See dataclass for meanings of args.

Variables​

  • static date_column : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static num_latest : int
  • static remote_modeller : bool

NumericThresholdFilterArgs​

class NumericThresholdFilterArgs(*args, **kwargs):

Arguments for numeric lower and upper metadata thresholds.

Variables​

  • static flag_column_name : str | None
  • static flag_only : bool
  • static maximum_values_by_column : dict[str, int | float]
  • static minimum_values_by_column : dict[str, int | float]
  • static remote_modeller : bool

PatientIDFilterArgs​

class PatientIDFilterArgs(*args, **kwargs):

Arguments for PATIENT_ID filter strategy.

This strategy reads a list of patient MRNs (or patient IDs) from a CSV file and excludes records from the dataframe that match those MRNs.

Arguments

  • filename: Path to the CSV file containing patient MRNs/IDs to exclude.
  • patient_id_column: Column name in the exclusion CSV file that contains the patient MRNs/IDs to exclude. This is NOT the column in the dataframe being filtered.

Variables​

  • static flag_column_name : str | None
  • static flag_only : bool
  • static patient_id_column : str
  • static remote_modeller : bool

RecordFilterAlgorithm​

class RecordFilterAlgorithm(    datastructure: DataStructure,    strategies: Sequence[FilterStrategy | str],    filter_args_list: list[FilterArgs],):

Algorithm factory for filtering records based on various strategies.

Arguments

  • **kwargs: Additional keyword arguments.
  • datastructure: The data structure to use for the algorithm.
  • filter_args_list: List of strategy-specific arguments
  • strategies: List of filtering strategies

Attributes

  • class_name: The name of the algorithm class.
  • datastructure: The data structure to use for the algorithm
  • fields_dict: A dictionary mapping all attributes that will be serialized in the class to their marshmallow field type. (e.g. fields_dict = {"class_name": fields.Str()}).
  • filter_args_list: List of strategy-specific arguments
  • nested_fields: A dictionary mapping all nested attributes to a registry that contains class names mapped to the respective classes. (e.g. nested_fields = {"datastructure": datastructure.registry})
  • strategies: List of filtering strategies

Ancestors​

Variables​

  • static fields_dict : ClassVar[T_FIELDS_DICT]

Methods​

create​

def create(self, role: str | Role, **kwargs: Any) ‑> Any:

Create an instance representing the role specified.

modeller​

def modeller(    self, *, context: ProtocolContext, **kwargs: Any,) ‑> bitfount.federated.algorithms.filtering_algorithm._ModellerSide:

Inherited from:

BaseNonModelAlgorithmFactory.modeller :

Modeller-side of the algorithm.

worker​

def worker(    self, *, context: ProtocolContext, **kwargs: Any,) ‑> bitfount.federated.algorithms.filtering_algorithm._WorkerSide:

Inherited from:

BaseNonModelAlgorithmFactory.worker :

Worker-side of the algorithm.

RequiredColumnPresentFilterArgs​

class RequiredColumnPresentFilterArgs(*args, **kwargs):

Arguments for requiring populated metadata columns on each record.

Variables​

  • static column_names : list[str]
  • static flag_column_name : str | None
  • static flag_only : bool
  • static remote_modeller : bool

ScanFrequencyFilterArgs​

class ScanFrequencyFilterArgs(*args, **kwargs):

Arguments for SCAN_FREQUENCY filter strategy.

This filtering strategy keeps only patients with a minimum specified number of scans per year over a specified number of years.

Variables​

  • static date_column : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static min_number_of_scans_per_year : int
  • static number_of_years : int
  • static remote_modeller : bool

ScanInPeriodFilterArgs​

class ScanInPeriodFilterArgs(*args, **kwargs):

Arguments for SCAN_IN_PERIOD filter strategy.

Keeps only patients with at least one scan within a specified time period. When laterality_column is provided, both a left AND a right scan are required; otherwise any scan in the period qualifies. All records for qualifying patients are retained.

The period can be specified in one of three mutually exclusive ways:

  • last_n_months: relative period, e.g. last 12 months from today
  • last_n_years: relative period, e.g. last 1 year from today
  • start_date + end_date: absolute date range (ISO format or parseable string)

Arguments

  • date_column: Column containing scan/acquisition dates.
  • id_column: Column(s) to group patients by.
  • laterality_column: Column containing eye laterality (e.g. "L"/"R"). When omitted, the both-eyes requirement is skipped.
  • last_n_months: Number of months back from today for relative period.
  • last_n_years: Number of years back from today for relative period.
  • start_date: Start of absolute date range (inclusive).
  • end_date: End of absolute date range (inclusive).

Variables​

  • static date_column : str
  • static end_date : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static last_n_months : int
  • static last_n_years : int
  • static laterality_column : str
  • static remote_modeller : bool
  • static start_date : str

SeriesDescriptionLatestFilterArgs​

class SeriesDescriptionLatestFilterArgs(*args, **kwargs):

Arguments for SERIES_DESCRIPTION_LATEST filter strategy.

This filtering strategy keeps only the latest records per ID and scan type, inferred from the series description.

Variables​

  • static columns_column : str
  • static date_column : str
  • static flag_column_name : str | None
  • static flag_only : bool
  • static id_column : str | list[str]
  • static manufacturer_column : str
  • static number_of_frames_column : str
  • static remote_modeller : bool
  • static rows_column : str
  • static series_description_column : str