Skip to main content

functions

Pure functions for EHR querying.

Module​

Functions​

apply_ehr_write_guard​

def apply_ehr_write_guard(    cache: CacheProtocol,    task_hash: str,    records: list[EHRDataRecord],) ‑> tuple[list[EHRDataRecord], list[EHRDataRecord]]:

Split records into those safe to write and those that would destroy data.

ehr_data holds exactly one row per (task_hash, bitfount_patient_id) and bulk_insert_ehr_data merges a fully-populated instance, so every write is a destructive full-row replace: a None overwrites a good stored value, and there is no history table and no versioning to recover it from. A patient whose fetch failed this run therefore has to be kept away from the write, not merely written carefully — which is what makes the fallback the rest of this project relies on real, since reads are partition-wide and merge never deletes.

A record is a failure record when its error is set. Both builders in this module set it: on a negative record for a lookup that failed outright, and on a built record whose every countable fetch group failed (_every_supported_group_failed — which is narrower than "every group", and narrower still than "every fetch"). This function does not re-derive that verdict — error is not None is the discriminator.

warning

This guard protects the total-failure case only. A fetch that retrieved some group still merges, so it can replace a stored row that held more (see the coverage-comparison note below). "A failed fetch cannot destroy stored data" is therefore not what this does.

  • error is None — always written. A successful fetch is this run's answer.
  • error set — written only when neither a stored row nor another record in this same batch already answers for the patient. With either, that is the better answer and this run's failure is dropped; with neither, the failure is worth recording, since the absence of a row cannot say whether the patient was ever queried.

The same-batch half matters because one patient can produce two records in one run: processed_ids is keyed on the imaging query key, not on BitfountPatientID, so a person reached under two imaging patient IDs is fetched twice, and drop_ambiguous_patients leaves that case alone (the EHR is not asserting they are different patients). If one fetch succeeded and the other's every group failed, both records reached the write with the same primary key and bulk_insert_ehr_data merges per record — so the one written last decided the stored row, and a failure could erase a success retrieved seconds earlier. The imaging route resolves this for lookup failures already (built_patient_ids against failed_lookups); a group failure inside a successful lookup never reached that check.

Arguments

  • cache: The run's cache, probed for stored rows.
  • task_hash: The partition in scope. A row under another task_hash is not a fallback for this one.
  • records: The records about to be written, after drop_ambiguous_patients.

Returns A (writable, withheld) pair, each in input order: the records to hand to bulk_insert_ehr_data, and the failure records dropped in favour of a stored row or of this run's own successful one.

drop_ambiguous_patients​

def drop_ambiguous_patients(    records: list[EHRDataRecord],) ‑> tuple[list[EHRDataRecord], set[str]]:

Drop records whose BitfountPatientID maps to more than one EHR patient.

ehr_data is keyed on (task_hash, bitfount_patient_id), and the BitfountPatientID is derived from name + date of birth. Two distinct EHR patient records therefore collapse onto one cache row — either one patient duplicated in the EHR, or two different patients who share a name and date of birth. Nothing available at this point tells those two cases apart, and the session.merge in bulk_insert_ehr_data would silently attribute one patient's codes and appointment history to the other, so the whole BitfountPatientID is excluded instead.

This mirrors the name+DOB lookup paths, where an EHR search matching more than one patient raises NonSpecificPatientError and the patient is skipped rather than guessed at. Excluding a patient costs an enrolment; guessing risks assessing one patient's eligibility against another patient's clinical data.

Records that share a BitfountPatientID and an ehr_patient_id are one EHR patient reached more than once (e.g. two imaging patient IDs for the same person) and are left alone — the EHR itself is not asserting they are different patients.

Arguments

  • records: Assembled records, before they are written to the cache.

Returns A (kept, ambiguous) pair: the records for every unambiguous BitfountPatientID in input order, and the set of BitfountPatientIDs that were dropped. The caller deletes any row already stored under an ambiguous ID — an earlier run may have written one before the second EHR record was known, and that row cannot be trusted to belong to either patient.