Skip to main content

v2

configured_datasources record, v2 — canonical aliases.

v2 adds the nullable indexing-order columns config_last_updated, archived_only and health_status — see schema.py.

Module​

Submodules​

Functions​

downgrade​

def downgrade(op: Operations) ‑> None:

Migrate configured_datasources v2 -> v1 (see the module docstring).

upgrade​

def upgrade(op: Operations) ‑> None:

Migrate configured_datasources v1 -> v2 (see the module docstring).

Classes​

Record​

class Record(**data: Any):

One datasource the pod has configured, or used to.

Attributes

  • task_hash: The datasource leaf hash, as file_metadata and runs carry it.
  • pod_name: Pod the datasource belongs to.
  • datasource_name: Name of the datasource within that pod.
  • deconfigured_at: When the datasource stopped being in the pod config. None while it still is.
  • last_seen_at: When the pod last took a snapshot covering this row.
  • config_last_updated: See DatasourceMetadata.
  • archived_only: See DatasourceMetadata.
  • health_status: See DatasourceMetadata.
  • tags: Arbitrary flat metadata.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Variables​

  • static archived_only : bool | None
  • static datasource_name : str
  • static health_status : str | None
  • static model_config
  • static pod_name : str
  • static task_hash : str

ORM​

class ORM(**kwargs):

SQLAlchemy model for the configured_datasources cache table.

One row per datasource the pod has ever had configured, recording whether it is configured now. Written as a whole-set snapshot by the pod, which is the only process that knows the pod config; read by processes that hold the cache but not the config — chiefly file_metadata_refresh, which runs in the orchestrator and would otherwise re-index a datasource that was removed from the pod weeks ago.

A removed datasource is tombstoned rather than deleted, and its file_metadata rows are left alone: those rows are keyed by file_path alone and are shared inventory (see file_metadata/v1/store.py's prune_missing_under_root), so a delete scoped by task_hash would blind a surviving datasource whose root overlaps.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Variables​

  • archived_only : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • config_last_updated : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • datasource_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • deconfigured_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • health_status : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • last_seen_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • pod_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • tags : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • task_hash : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]