PII redaction: build it properly, then test it

TL;DR: PII redaction finds and masks personal data in text, logs and transcripts so a system still works without exposing anyone. Put it at ingestion, not on export. Pick the technique by how much data use you need to keep. Then test it against the EDPB's three criteria: no record isolation, no linkage, no inference. Untested redaction is a guess.

Most teams run a regex over their logs, mask the obvious fields, and call the data anonymous. It isn't. Masking a value and making someone unidentifiable are two different jobs. Only one of them holds up in an audit.

Here's the thing. Where you put redaction in the pipeline matters more than which masking technique you pick. Redact at ingestion and the raw data never lands in your primary store. Redact on export and it sits there forever, waiting.

This guide covers what redaction actually is, what needs redacting, which technique fits, how to build it for logs and transcripts, and the three tests that prove it worked.

Redaction, anonymisation and pseudonymisation are not the same thing

Teams use these words like they mean the same thing. They don't, and regulators know the difference.

• Redaction removes or masks specific values when they are captured or processed. It's an operational control. It's not a legal guarantee.
• Pseudonymisation swaps identifiers for tokens but keeps a map somewhere. The data can still be traced back. Regulators treat it as a security measure, not real anonymity.
• Anonymisation means re-identification isn't practical, even when someone joins your data to another dataset.

The EDPB's anonymisation guidelines set three tests. Learn them.

1. No record isolation. You can't pick one person out of the data.
2. No linkage. You can't join records across datasets back to the same person.
3. No inference. You can't work out facts about someone with high confidence.

Redaction on its own rarely passes all three. So keep the word "anonymised" for data you have actually tested. Spanish regulator guidance goes further and treats pseudonymised data as personal data, still under the full protection rules.

What actually needs redacting

Start with the data that carries direct risk. Full names. Phone numbers. Email addresses. Government IDs. Card and bank details. Health records.

Then deal with quasi-identifiers. These look harmless on their own. Postcode. Date of birth. Job title. Put three of them together and you can often pick one person out of a dataset of thousands.

Now here's the important bit. Scope your controls to where the data moves, not to a table.

• At ingestion. Redact before the data hits your main store. Best default for anything new.
• At rest. Mask existing stores you can't rebuild yet.
• On export. Redact when data leaves for analytics or a third party.

Export-only redaction feels easy. It also leaves raw records sitting in your database for years. If you can only build one layer, build it at ingestion.

Which technique fits

It comes down to one trade off. How much use do you need to keep, against how much re-identification risk you can live with.

Most cloud services now treat these as policies you configure, not one fixed behaviour. Azure's Language service supports character mask, entity mask, synthetic replacement, and a confidence threshold you can tune per entity type. So you can loosen detection on low-risk fields and tighten it on financial or health data. The redaction policy types are set out in Microsoft's docs.

Add an exclusion list for known false positives. A company name that reads like a person's name will trip your detector every time. One list kills most of the noise.

Logs, structured data and transcripts need different code

This is where most builds break. Structured data and free text are different problems, and one detector won't do both well.

For JSON and structured logs, walk the object tree and return a new object. Don't edit in place. Mask the values, leave the keys and the shape alone. Naive in-place edits corrupt anything downstream that expects the original schema. Config-driven adapters that mask fields before data reaches a model usually work this way, with reversible token maps scoped to a session, or irreversible modes where you never need the value back.

For application logs, set the logger itself to redact known sensitive fields. API keys, tokens, session IDs. Don't rely on a scrubber further down the line, and don't log secrets at all.

For transcripts and free text, pair deterministic detectors for known formats with an NER model for names and addresses. Streaming redaction has to keep timestamps and offsets intact so the text still lines up with the audio. Batch redaction has room for a second pass.

One problem though. Automated redaction on transcripts misses things. Amazon's own documentation says so, and recommends human review or keeping an unredacted copy under strict access controls when you may need to check what was actually said. If you keep that copy, put real role-based access control around it.

The three tests that prove it worked

A redaction policy nobody has tested is a guess. Test it with the same three criteria the regulators use.

1. Isolation test. Pull a random redacted record. Try to identify the person from what is left.
2. Linkage test. Try to join the redacted set to a public or internal dataset using the quasi-identifiers still in there.
3. Inference test. Check whether health status, income or similar can be guessed from the surrounding context.

Then run the boring engineering checks. Unit test your detectors against a labelled sample. Track false positives and false negatives separately, because they need different fixes. Watch the confidence scores your detector hands out. A cluster of scores sitting right on your threshold almost always means a category that needs its own rule.

Re-identification methods get better every year. So this isn't a one-off exercise. Put the tests in your release process and run them again on a schedule.

What a real pipeline looks like end to end

Enough theory. Here is the order of operations we build to.

1. Classify first. You can't redact what you haven't found. Map every field and every free-text blob to a category before you write a detector. Most teams skip this and spend the next year patching gaps.
2. Detect at the edge. Run deterministic detectors on structured fields as data arrives. Regex and format checks are cheap and predictable. Use them where the shape is known.
3. Layer NER over free text. Names and addresses don't have a format. That's the model's job, not a regex's job.
4. Apply the policy per field. Not one blanket rule. A support ticket body and a payment reference need different handling.
5. Write the audit row. What was detected, what policy fired, what confidence, which version of the detector. No audit row means no answer when someone asks.
6. Store the token map separately. If the mapping lives in the same database as the redacted data, you haven't reduced risk. You have moved it.
7. Test on the way out. Run the three tests against a sample before the data leaves for analytics or a third party.

Step 5 is the one teams cut for time. Then a security review lands and nobody can say which records were processed under which policy. Rebuilding that history after the fact isn't possible.

Four mistakes we see over and over

Redacting only on export. The raw data is still there. A breach of the primary store gives up everything.

Trusting one detector. A single regex pass catches the formatted stuff and misses every name in a free-text note.

Treating a threshold as set and forget. Detector confidence drifts as your data changes. Watch it like you watch error rates.

Storing the token map next to the data. If one credential opens both, the tokens are decoration.

Fixing any of these is cheap early and expensive later. That's true of most data work. It's very true here, because the cost lands as a disclosure, not a bug.

Where redaction falls over

No system catches everything. Pretending otherwise is how teams get surprised.

False positives over-redact and break downstream logic. False negatives leave real data exposed. Both happen far more on messy free text than on clean fields.

The bigger risk is inference. Research in Science Advances on the imperfect science of anonymisation shows reconstruction attacks can pull sensitive records back out of supposedly anonymous sets. Language models can also repeat sequences from their own training data. That's why layers beat any single technique.

• Keep raw data only as long as the business actually needs it.
• Lock down anything unredacted.
• Set masking rules by role, not one blanket rule for everyone.

Redaction is one checkpoint. It sits alongside the others in a proper LLM guardrails architecture, and it needs the policy layer above it, which is what data governance for AI covers.

How we build it at Devwiz

We treat redaction as infrastructure. It goes in at the architecture stage, not as a patch before launch.

That means immutable recursive traversal for structured data. Reversible token maps only where a workflow genuinely needs the value back. Audit logging on every redaction decision, so a security review has something real to look at.

Devwiz has shipped 200+ applications, including work for the NSW Government (Justice and Corrective Services), Briometrix, Vivid and Huskee. Enough builds to have seen how data handling goes wrong at every size.

Weighing up an off-the-shelf library against a purpose-built pipeline? The deciding factor is usually scale and audit requirements, not budget. A library is fine when nobody will ever ask you to prove what was redacted and when. Build it properly when they will.

The gap between a checkbox and a real pipeline

Pick a masking library, ship it, move on. That advice isn't wrong. It's just thin.

Where you place redaction beats which technique you choose, every time. Redacting on export while raw data sits untouched at rest looks fine on a diagram. It fails the first serious audit.

Plus the rules keep moving. The EU AI Act compliance requirements landed this year and they change what "we handle data carefully" has to mean in writing. There's a related argument about owning your data rather than just your model, which is really the same point one layer up. And if you're embedding text for search, remember the vectors carry the content too, so read up on what a vector database actually stores before you assume redaction upstream covered you.

If you do one thing, do this. Put the re-identification test in your release process from day one. Not after an incident. Teams that treat redaction as a setting rather than a monitored control are the ones who get caught out. Broader context on the rules sits in our guide to AI software compliance.

Need this built properly, not bolted on?

If your platform handles support transcripts, health data or financial records, the redaction work starts at architecture.

We build custom platforms and the data pipelines underneath them, including the redaction logic, audit trails and token management. Have a look at AI app development, AI programs, or turning your program into a platform.

Worth a chat?

Frequently asked questions

What does PII redaction mean?

PII redaction is finding and removing or masking personally identifiable information, such as names, emails or ID numbers, from text, logs or transcripts before that data is stored, shared or processed further.

What PII should be redacted?

Start with names, phone numbers, email addresses, government and financial IDs, and health data. Then handle quasi-identifiers like postcode and date of birth, which become identifying once you combine them with other fields.

How do you redact PII in practice?

Run deterministic detectors on structured fields at ingestion, layer a named-entity recognition model over free text and transcripts, apply the policy per field, and log every redaction decision. Then test the result against the no-isolation, no-linkage, no-inference framework before you treat it as safe.

Is pseudonymised data the same as anonymised data?

No. Pseudonymisation swaps identifiers for tokens but keeps a map somewhere, so regulators still treat it as personal data under the full protection rules. Anonymisation means re-identification is not practical at all.

Where should redaction sit in the pipeline?

At ingestion, so raw personal data never lands in your primary store. Export-time masking is a useful second layer, but on its own it leaves the raw records sitting in your database indefinitely.

How do you test a redaction policy?

Use the three tests regulators use. Try to identify a person from a single redacted record. Try to join the redacted set to another dataset using the quasi-identifiers left behind. Try to infer sensitive attributes from the surrounding context. Run all three on a schedule, not once.