Skip to main content

download_utils

Utilities for downloading files from URLs.

Module​

Functions​

download_file_from_url​

def download_file_from_url(    url: str,    target_path: Path,    *,    decompress_gzip: bool = True,    skip_if_exists: bool = True,) ‑> pathlib.Path:

Download a file from a public URL to a local path.

Supports gzip decompression when the URL ends with .gz. Skips download if the target file already exists (when skip_if_exists is True).

Arguments

  • url: The URL to download from (e.g. public S3 URL or any HTTP(S) URL).
  • target_path: Local path where the file will be saved.
  • decompress_gzip: If True and URL ends with .gz, decompress before saving. Defaults to True.
  • skip_if_exists: If True and target_path exists, skip download. Defaults to True.

Returns The path to the downloaded (or existing) file.

Raises

  • requests.HTTPError: If the response code is not 200.
  • requests.RequestException: If the request fails.