Skip to main content

schema

SQLAlchemy ORM for the flow_specs cache table (v1).

Each version package declares its own local Base (its own MetaData) so that multiple versions of the same table can coexist in one process without a __tablename__ collision. Base carries the shared nullable tags column and every ORM in this version inherits it directly.

Adding a column to a new version is an additive migration (see migrations.py): add a nullable mapped_column in the new version's schema and a corresponding upgrade step — never rename/drop/retype in place.

Classes​

Base​

class Base(**kwargs: Any):

Local declarative base for flow_specs v1.

Owns an isolated MetaData (so it never collides with another version of this table) and inherits the shared columns from types/schema.py's Base.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Subclasses​

Variables​

  • static metadata
  • static registry

FlowSpecRow​

class FlowSpecRow(**kwargs):

SQLAlchemy model for the flow_specs cache table.

Holds the replay material for one (project_id, datasource_name): the FlowSpec exactly as the Hub sent it, plus the lineage bookkeeping recovery needs. Without this row a crashed background run cannot be rebuilt, because the FlowSpec exists nowhere else on the pod — it arrives inside the DATASET_PROJECT_LINKED message and was previously discarded once parsed.

One row per (project_id, datasource_name), upserted on every dataset-project link: the latest link wins, and re-linking is therefore the natural reset for a lineage that gave up.

The table also acts as the discriminator for recovery. Only the DATASET_PROJECT_LINKED path writes it, so "is this a background DAG run?" is answered by joining against this table rather than by pattern-matching the free-form type column in runs, which four different writers share.

Note the deliberate asymmetry with runs: this table describes a trigger and its lineage, not an execution. There is exactly one row per linked (project, datasource) however many runs that link produces.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Variables​

  • dag_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • datasource_name : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • flow_spec_raw : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • gave_up_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • gave_up_reason : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • linked_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • origin_task_id : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • project_id : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • tags : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]