Skip to main content

metadata

Datasource metadata fetching for preprocessing transforms.

Some transforms need datasource columns that are not part of the model output (e.g. Columns/Rows to scale fovea landmark coordinates). fetch_postprocessor_metadata reads those rows so a calc step can supply them to apply_preprocessing via its fetch_metadata callback.

Module​

Functions​

fetch_postprocessor_metadata​

def fetch_postprocessor_metadata(    datasource: BaseSource, keys: list[str], columns: list[str],) ‑> pandas.core.frame.DataFrame | None:

Get datasource rows used to supply postprocessor metadata columns.

For a FileSystemIterableSource this passes skip_non_tabular_data=True so only the tabular metadata (e.g. Columns/Rows for the fovea landmark scaling) is loaded rather than full scan payloads, and degrades gracefully to None rather than failing the consuming step.

Arguments

  • datasource: The datasource to read metadata rows from.
  • keys: The file keys (ORIGINAL_FILENAME_METADATA_COLUMN values) to fetch rows for.
  • columns: The metadata columns required by the postprocessors (used for logging context only; the merge is performed by the caller).

Returns A metadata DataFrame aligned to keys, or None when the datasource cannot supply it.