Skip to main content

types

Datasource related types.

Classes​

DataSourceFileFilter​

class DataSourceFileFilter(*args, **kwargs):

A datasource-level filter that can be applied to files.

Provides a datasource-level filter (i.e. one that does not rely solely on filesystem information, but instead on contents of the data) that can be applied to files to include/exclude it from a datasource.

Ancestors​

DatasourceSummaryStats​

class DatasourceSummaryStats(*args, **kwargs):

Dictionary with datasource summary statistics.

Ancestors​

  • builtins.dict

Variables​

  • static additional_metrics : dict[str, typing.Any]
  • static files_with_errors : int
  • static skip_reasons : Dict[str, int]
  • static total_files_found : int
  • static total_files_skipped : int
  • static total_files_successfully_processed : int

Date​

class Date(year: int, month: Optional[int] = None, day: Optional[int] = None):

Simple date class used for filtering files based on date headers.

This is used by FileSystemIterableSource to filter files based on their creation and modification dates.

Arguments

  • year: The oldest possible year to consider.
  • month: The oldest possible month to consider. If None, all months in the given year are considered. Defaults to None.
  • day: The oldest possible day to consider. If None, all days in the given month are considered. If month is None, this is ignored. Defaults to None.

Ancestors​

  • bitfount.types.UsedForConfigSchemas

Variables​

  • static day : Optional[int]
  • static month : Optional[int]
  • static year : int

Methods​

get_date​

def get_date(self) ‑> datetime.date:

Get a datetime.date object from the date components year, month and day.

DateTD​

class DateTD(*args, **kwargs):

Typed dict form of Date dataclass.

Ancestors​

  • builtins.dict

Variables​

  • static day : typing_extensions.NotRequired[int]
  • static month : typing_extensions.NotRequired[int]
  • static year : int