Skip to main content

hugging_face_perplexity

Hugging Face Perplexity Algorithm.

Reference: https://huggingface.co/docs/transformers/perplexity#example-calculating-perplexity-with-gpt2-in-transformers

Classes​

HuggingFacePerplexityEvaluation​

class HuggingFacePerplexityEvaluation(    model_id: str, text_column_name: str, stride: int = 512, seed: int = 42,):

Hugging Face Perplexity Algorithm.

Arguments

  • model_id: The model id to use for evaluating its perplexity. The model id is of a pretrained model hosted inside a model repo on huggingface.co. Accepts models with a causal language modeling head.
  • seed: Sets the seed of the algorithm. For reproducible behaviour it defaults to 42.
  • stride: Sets the stride of the algorithm. Defaults to 512.
  • text_column_name: The single column to query against. Should contain text for generation.

Attributes

  • class_name: The name of the algorithm class.
  • fields_dict: A dictionary mapping all attributes that will be serialized in the class to their marshamllow field type. (e.g. fields_dict = {"class_name": fields.Str()}).
  • model_id: The model id to use for evaluation. The model id is of a pretrained model hosted inside a model repo on huggingface.co. Accepts models with a causal language modeling head.
  • nested_fields: A dictionary mapping all nested attributes to a registry that contains class names mapped to the respective classes. (e.g. nested_fields = {"datastructure": datastructure.registry})
  • seed: Sets the seed of the algorithm. For reproducible behaviour it defaults to 42.
  • stride: Sets the stride of the algorithm. Defaults to 512.
  • text_column_name: The single column to query against. Should contain text for generation.

Ancestors​

Variables​

Methods​

create​

def create(self, role: Union[str, Role], **kwargs: Any) ‑> Any:

Create an instance representing the role specified.

modeller​

def modeller(    self, **kwargs: Any,) ‑> bitfount.federated.algorithms.hugging_face_algorithms.base._HFModellerSide:

Returns the modeller side of the HuggingFacePerplexityEvaluation algorithm.

worker​

def worker(    self,    **kwargs: Any,) ‑> bitfount.federated.algorithms.hugging_face_algorithms.hugging_face_perplexity._WorkerSide:

Returns the worker side of the HuggingFacePerplexityEvaluation algorithm.