Text Redaction

DICOM description fields frequently contain free-text PHI. Operators populate SeriesDescription, StudyDescription, and ProtocolName, which can contain names, dates, accession numbers, and other identifiers. dicom-dre redacts these fields token by token against an allowlist, so known clinical vocabulary passes through and the engine masks unrecognized tokens.

Important

Intended scope. The redactor targets short, semi-structured description fields such as SeriesDescription and StudyDescription. In these fields the distinct values are overwhelmingly derived from structured entries in the source PACS (protocol names, modality codes, body parts, view labels), so a finite allowlist covers the legitimate vocabulary and unrecognized tokens are strong PHI candidates.

Do not use the redactor on paragraphs of prose, general clinical notes, report bodies, or any field that is entirely free-form text with no discernible pattern. The token-allowlist model does not bound the vocabulary of such fields and cannot provide reliable de-identification for them.

Note

The bundled free-text allowlist comes from studies on a single PACS at one medical research center. Its vocabulary reflects the reporting conventions seen there. It is unlikely to be complete or correct for another site. Validate and extend the allowlist against your own descriptions before relying on it, to avoid over-redaction (masking legitimate terms) and under-redaction (leaking PHI). See Limitations and portability and Local validation.

How redaction works

Redaction runs in three passes over each field value:

  1. Pattern masking. Regular expressions match structured content that the redactor treats as sensitive regardless of the allowlist: dates, times (unless dates are preserved), email addresses, URLs, hexadecimal strings of six or more characters, US phone numbers, Social Security numbers, ZIP+4 codes, ZIP codes preceded by a US state, and the prefixes NRP, MRN, and SSN followed by four or more digits. The redactor replaces matched characters with X and leaves separators such as /, -, ., :, and whitespace in place, so the value retains its width and separator layout.

  2. Tokenization. The redactor splits the remaining text on a fixed set of delimiters (spaces, punctuation, and boundaries between letters and digits).

  3. Allowlist filtering. The redactor keeps each token when its lowercased form is in the allowlist or when it matches an allow pattern (for example a one-to-five digit number). It replaces any other token with a run of X characters of equal length. It leaves already-masked tokens unchanged.

Because masked tokens keep the length of the source token, the redacted value retains its field width and separator layout while dropping the identifying content.

Date preservation

The preserve_dates flag controls whether the redactor retains dates and times:

  • When preserve_dates is False (default and strict profiles), the redactor masks date and time tokens along with everything else off the allowlist.

  • When preserve_dates is True (lds and lds-no-dob profiles), the redactor skips the date-masking and time-masking passes and adds month names, four-digit years, and timezone abbreviations to the allow patterns so they pass through intact. It still masks emails, URLs, and hexadecimal strings.

See De-identification Profiles for the per-profile flag values.

Allowlist source

The allowlist is a CSV of permitted tokens loaded from the package resources. Each profile names its allowlist through the allowlist_csv attribute (default default.csv). A caller can substitute a custom allowlist by supplying a different filename or an absolute path. The redactor CLI commands operate on CSV files using the same allowlist mechanism and support curating the list; see the Redactor guide.

Description-field behavior

For the description fields (SeriesDescription, StudyDescription, ProtocolName), a caller-supplied value takes precedence: the engine writes it verbatim with no redaction. When the caller supplies no value, the engine redacts the field present in the source dataset with the profile’s allowlist and preserve_dates setting.

Annotation and structured-content free text

The default profile applies the same redaction to the free-text attributes of the retained annotation and Key Object Selection subtrees:

  • Unformatted Text Value (0070,0006) and Tick Label (0070,0289) in the Graphic Annotation Sequence of admitted 2D softcopy presentation states.

  • Text Value (0040,A160) in the Content Sequence of admitted KO documents.

Each is bound to an explicit rule against the profile allowlist and preserve_dates setting. The redaction action decodes raw bytes and resolves the dictionary VR for an implicit-VR (UN/OB) element, so redaction is independent of encoding. See De-identification Profiles.