Skip to main content

post_processing_base

Base classes and utilities for post-processors.

Module​

Functions​

apply_postprocessors​

def apply_postprocessors(postprocessors: list[PostProcessor], predictions: Any) ‑> Any:

Apply a list of postprocessors to predictions in sequence.

Arguments

  • postprocessors: List of postprocessors to apply.
  • predictions: Model predictions to process.

Returns Processed predictions

create_postprocessor​

def create_postprocessor(    config: dict[str, Any], access_token: str | None = None,) ‑> PostProcessor | None:

Create a postprocessor from a configuration dictionary.

Arguments

  • config: Postprocessor configuration with 'type' and other parameters
  • access_token: An optional HuggingFace access token

Returns An instance of the requested postprocessor or None if there is an error when creating it.

create_postprocessors​

def create_postprocessors(    postprocessor_configs: list[dict[str, Any]] | None = None,    access_token: str | None = None,) ‑> list[PostProcessor]:

Create a list of postprocessors from configurations.

Arguments

  • postprocessor_configs: Configuration for postprocessors, either:
  • None (returns an empty list)
  • A list of dicts, each with 'type' and other parameters
  • access_token: An optional HuggingFace access token

Returns List of PostProcessors.

get_matching_columns​

def get_matching_columns(all_columns: list[str], patterns: list[str]) ‑> list[str]:

Get column names matching the given patterns.

Arguments

  • all_columns: List of all column names.
  • patterns: List of regex patterns to match.

Returns List of column names that match at least one pattern.

get_postprocessor_metadata_columns​

def get_postprocessor_metadata_columns(    postprocessor_configs: list[dict[str, Any]] | None,) ‑> tuple[str, ...]:

Return unique metadata columns requested by postprocessor configs.

Reads each config's columns_required entry. These columns (e.g. Columns/Rows for fovea_v7_to_v6) must be present on the predictions DataFrame before the postprocessors run.

parse_json​

def parse_json(value: Any) ‑> Any:

Parse JSON data from various formats.

Arguments

  • value: The value to parse.

Returns Parsed JSON object or original value if parsing fails.

preserve_postprocessor_metadata_columns​

def preserve_postprocessor_metadata_columns(    output: PredictReturnType | pd.DataFrame,    *,    metadata_columns: tuple[str, ...],    fetch_metadata: MetadataFetcher,    keys: Sequence[str] | None = None,) ‑> PredictReturnType | pandas.core.frame.DataFrame:

Merge metadata columns required by postprocessors into the predictions.

Any column in metadata_columns that is absent from the predictions is fetched via fetch_metadata (keyed by ORIGINAL_FILENAME_METADATA_COLUMN) and aligned onto the predictions. Predictions are returned unmodified when no columns are missing or the metadata cannot be retrieved/aligned.

Arguments

  • output: Predictions as a PredictReturnType or DataFrame.
  • metadata_columns: Columns required by the postprocessors (see get_postprocessor_metadata_columns).
  • fetch_metadata: Callback returning datasource rows for the requested keys/columns, or None when unavailable.
  • keys: Prediction keys (filenames) used to align fetched metadata. Required when output is a DataFrame without the filename column.

Classes​

CompoundPostProcessor​

class CompoundPostProcessor(    postprocessing_sequence: list[dict[str, Any]], name: str | None = None,):

Applies multiple postprocessors in sequence.

Initialize the compound postprocessor.

Arguments

  • postprocessing_sequence: List of postprocessor configurations to apply in a sequence
  • name: Name as a string for the for this transformation pipeline. Default to None.

Ancestors​

Methods​

process​

def process(self, predictions: Any) ‑> Any:

Apply all postprocessors in sequence.

PostProcessor​

class PostProcessor():

Base class that all postprocessors inherit from.

Subclasses​

Methods​

process​

def process(    self, predictions: PredictReturnType | pd.DataFrame,) ‑> PredictReturnType | pandas.core.frame.DataFrame | typing.Any:

Process the model predictions.

PostprocessorType​

class PostprocessorType(*args, **kwds):

Types of built-in postprocessors.

Ancestors​

Variables​

  • static COMPOUND
  • static HUGGINGFACE_APPLY_ID_TO_LABELS
  • static JSON_EXTRACT_TO_COLUMNS
  • static JSON_KEY_RENAME
  • static JSON_RESTRUCTURE
  • static JSON_WRAP_IN_LIST
  • static NER_DEIDENTIFICATION
  • static RENAME
  • static SCALE_LANDMARK_COORDINATES
  • static STRING_TO_JSON
  • static TRANSFORM
  • static ZIP_FLATTEN_COORDINATES