Skip to main content

schema

SQLAlchemy ORM for the configured_datasources cache table (v2).

v2 adds what the pod knows about each datasource when it decides the order metadata indexing runs in: the Hub's configurationLastUpdated, whether every project the datasource is linked to is archived, and the last health-check status. All three are nullable: None is "not known", which readers treat as "do not skip".

Each version package declares its own local Base (its own MetaData) so that multiple versions of the same table can coexist in one process without a __tablename__ collision.

Adding a column to a new version is an additive migration (see migrations.py): add a nullable mapped_column in the new version's schema and a corresponding upgrade step — never rename/drop/retype in place.

Classes​

Base​

class Base(**kwargs: Any):

Local declarative base for configured_datasources v2.

Owns an isolated MetaData (so it never collides with another version of this table) and inherits the shared columns from types/schema.py's Base.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Subclasses​

Variables​

  • static metadata
  • static registry

ConfiguredDatasourceRow​

class ConfiguredDatasourceRow(**kwargs):

SQLAlchemy model for the configured_datasources cache table.

One row per datasource the pod has ever had configured, recording whether it is configured now. Written as a whole-set snapshot by the pod, which is the only process that knows the pod config; read by processes that hold the cache but not the config — chiefly file_metadata_refresh, which runs in the orchestrator and would otherwise re-index a datasource that was removed from the pod weeks ago.

A removed datasource is tombstoned rather than deleted, and its file_metadata rows are left alone: those rows are keyed by file_path alone and are shared inventory (see file_metadata/v1/store.py's prune_missing_under_root), so a delete scoped by task_hash would blind a surviving datasource whose root overlaps.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Variables​

  • archived_only : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • config_last_updated : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • datasource_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • deconfigured_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • health_status : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • last_seen_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • pod_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • tags : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • task_hash : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]