utils

Utility functions for HuggingFace data.

Module

Functions

get_data_factory_dataset

def get_data_factory_dataset(    datasource: BaseSource,    data_split: DataSplit,    selected_cols: list[str],    selected_cols_semantic_types: Mapping[_SemanticTypeValue, list[str]],    batch_transforms: list[dict[str, _JSONDict]] | None,    data_splitter: DatasetSplitter | None = None,    labels2id: dict[str, int] | None = None,    target: str | list[str] | None = None,) ‑> tuple[_BaseHuggingFaceDataFactory, _IterableHuggingFaceDataset]:

Get the HuggingFace data factory and dataset for the given datasource.

---