Transformations

Batch transformations are handled by albumentations and are applied to images by the DataLoaders when the data is fed to the model. The default transformations are:

• Resize the image to 224x224 pixels
• Normalize: Normalize the image to ImageNet statistics
• ToTensorV2: Convert the image to a PyTorch tensor

These transformations are applied to the image data regardless of the task being performed (training, inference, evaluation). However the batch transformations can be customised to have different transformations for different segments of the data (training, validation, test). For training in particular it is important to have different transformations for the training and validation/test sets for augmentation purposes to avoid overfitting.

A full list of the available transformations is available in the Albumentations documentation.

Custom transformations

Transformations can be specified either in code or in YAML. In code, you can create a DataStructure object and pass the transformations to the batch_transforms argument. In the example below where we have a single image column, we have specified different transformations for the training and validation sets.

from bitfount.data import DataStructure

data_structure = DataStructure(
    selected_cols=["image_col", "target"],
    image_cols=["image_col"],
    target=["target"],
    batch_transforms=[
        {
            "albumentations": {
                "step": "train",
                "output": True,
                "arg": "image_col",
                "transformations": [
                    {"Resize": {"height": 224, "width": 224}},
                    {"HorizontalFlip": {"p": 0.5}},
                    "RandomBrightnessContrast",
                    "Normalize",
                    "ToTensorV2",
                ],
            }
        },
        {
            "albumentations": {
                "step": "validation",
                "output": True,
                "arg": "image_col",
                "transformations": [
                    {"Resize": {"height": 224, "width": 224}},
                    "Normalize",
                    "ToTensorV2",
                ],
            }
        },
    ],
)

In YAML, you can specify the transformations in the transform section of the task YAML file. The below YAML example specifies the exact same transformations as the code example above.

task:
  data_structure:
    select:
      include: ["image_col", "target"]
    assign:
      target: image_col
      image_cols: ["image_col"]
    transform:
      batch:
        - albumentations:
            step: train
            output: true
            arg: image_col
            transformations:
              - { "Resize": { "height": 224, "width": 224 } }
              - { "HorizontalFlip": { "p": 0.5 } }
              - "RandomBrightnessContrast"
              - "Normalize"
              - "ToTensorV2"
        - albumentations:
            step: validation
            output: true
            arg: image_col
            transformations:
              [
                { "Resize": { "height": 224, "width": 224 } },
                "Normalize",
                "ToTensorV2",
              ]

Multiple image columns

In cases where there are multiple image columns each with the same transformations, instead of specifying the transformations in batch_transforms individually, you can specify the transformations in the image_prefix_batch_transforms argument of the DataStructure object which will be applied to all image columns with the same prefix. In YAML, instead of specifying the transformations in the transform.batch section of the DataStructure, you can specify the transformations in the transform.image section of the DataStructure. In these cases, you will then omit the arg argument from the transformations.

Tips for reproducibility

• Keep the same pre and post processing for training, validation, and inference
• Augmentations should only be applied to the training set
• Make transformations deterministic when running inference and evaluation tasks i.e. no transformations that take a probability (p) as an argument