Skip to main content

v1

file_metadata record, v1 — canonical aliases.

Consumers and the record registry import from here and never touch the inner module names, mirroring steps/*/v1/__init__.py.

Module​

Submodules​

Classes​

ORM​

class ORM(**kwargs):

SQLAlchemy model for the file_metadata cache table.

Stores filesystem metadata for files in a datasource. Used to detect new or modified files without re-scanning the full datasource, and to support deduplication via content hashing.

A simple constructor that allows initialization from kwargs.

Sets attributes on the constructed instance using the names and values in kwargs.

Only keys that are present as attributes of the instance's class are allowed. These could be, for example, any mapped columns or relationships.

Ancestors​

Variables​

  • created_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • dataset_identifier : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • datasource_type : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • extension : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • file_hash : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • file_path : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • file_size_bytes : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • indexed_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • modified_at : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • row_number : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • run_id : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • source_path : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • tags : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]
  • task_hash : Union[sqlalchemy.orm.attributes.InstrumentedAttribute[+_T_co], +_T_co]

Record​

class Record(**data: Any):

Record of a single row from the file_metadata table.

Create a new model by parsing and validating input data from keyword arguments.

Raises [ValidationError][pydantic_core.ValidationError] if the input data cannot be validated to form a valid model.

self is explicitly positional-only to allow self as a field name.

Variables​

  • static dataset_identifier : str | None
  • static datasource_type : str | None
  • static extension : str | None
  • static file_hash : str | None
  • static file_path : str
  • static file_size_bytes : int | None
  • static indexed_at : str | None
  • static model_config
  • static row_number : int | None
  • static run_id : str | None
  • static source_path : str | None
  • static task_hash : str