🤖 Automation and Coffee: My Journey to Remove PII from Documents

Removing Personal Data from Scanned Documents Using AI and OCR: Challenges and Solutions for Privacy and Transparency

I found myself staring at a massive pile of documents, old contracts, legal cases, scanned PDFs, all filled with personal data. Manually removing that information one by one? Impossible without losing my mind. Trying to erase names and numbers with a PDF editor felt like using a bucket to empty a sinking ship. It’s not just about laziness: when you're dealing with large volumes, the task becomes unmanageable and highly prone to human error.

I’ve seen studies showing thousands of public documents leaking data because someone failed to redact properly. I had no intention of becoming part of that statistic.

It's pretty common to open a PDF that looks like personal data has been redacted, only to export the text and find everything wide open. That happens when people rely on non-professional tools that just add a “layer” over the document, instead of dealing with its actual structure.

And you don’t need to be a forensic expert to catch that.

I started this automation journey almost as a survival instinct. I thought: “What if a robot could do this for me?” With AI, I taught the machine to hunt down names, IDs, and addresses hidden in the pages, like a digital detective. First challenge: many documents were scanned as images, with no searchable text. The answer? OCR, basically giving eyes to the computer. It works surprisingly well. Where I saw a blurry stamp, the AI saw a full name. In minutes, the machine did what would take days of coffee and human effort, accurately identifying patterns like names and numbers. I felt a little joy watching the algorithm highlight sensitive sections on its own… goodbye, physical highlighters!

The great thing about using AI is that you can go beyond just catching obvious personal data like document numbers, you can train it to spot textual references that could reveal someone’s identity. For example, in a sentence like: “...and then the president of the Association said...,” there’s no name or number, but it’s clearly referring to a person. Removing that kind of personal data is much harder without AI, even advanced NLP techniques often miss it.

But the victory didn’t last long. Enter the 1984 dilemma: where was this AI processing everything? Uploading confidential files to the cloud? That felt like swapping one problem for another. I remembered the story of a tech giant banning ChatGPT internally after employees accidentally leaked trade secrets. The last thing I wanted was to make headlines for leaking data while trying to protect it. So I made a tough choice: go local.

Running AI models locally felt like bringing the party in-house. Security went up, but so did the power bill (and the server costs). Explaining the infrastructure expense was... fun (not really). Still, it beat handing over sensitive data to third parties. I found promising open-source tools, an OCR module here, a PII detector there, running everything in my own scrappy data center. It took effort, but sleeping at night knowing the data stayed within my control was worth a few late nights configuring GPUs. Google Cloud credits helped a lot, too.

As I tuned the system, I ran into a classic problem for public agencies: transparency vs. privacy. On one side, there’s pressure to publish contracts and legal docs for public oversight. On the other, you can’t expose addresses, phone numbers, or salaries. Once, we had to publish a simple procurement contract, but it contained the supplier's home address. If we published it as-is, we’d violate privacy. If we redacted too much, people would claim we were hiding something. That tightrope walk became routine. Automation came to the rescue: we started delivering “clean” docs, with redactions only where needed, striking a balance between open data and individual privacy.

I also learned that relying on people to manually redact thousands of pages is a recipe for disaster (and carpal tunnel). I’ve seen entire teams of well-meaning staff turn into zombies reviewing endless stacks of paper, inevitably missing an ID number here or a name there. Mistakes happen, a missed zero, a name that didn’t look like PII in the haze of exhaustion, and there goes confidentiality. Plus, the cost of all that human review adds up fast, with no guarantee of accuracy. Honestly, I’d rather reassign those brains to work that actually needs human judgment, and let machines handle the boring stuff, they don’t yawn or lose focus.

In the end, this journey taught me that privacy compliance goes far beyond updating a few fields in a database. It’s like opening a forgotten basement full of dusty boxes: beyond the official systems, there's an entire shadow archive of scattered data, what people call “shadow IT.” Spreadsheets on a controller’s desktop, old scanned contracts, even physical files covered in dust, and yes, they all contain personal information. Ignoring this dark side of IT is asking to be blindsided.

Today, with AI on my side, I face this challenge on multiple fronts. Technology isn’t a silver bullet, but it’s our best shot at outlasting the mess, because tech doesn’t get tired (just expensive). I’m still refining models, tweaking filters, and, yes, doing the occasional manual spot check to make sure nothing slips through. This experience taught me humility: you can’t just adopt a magical tool and call it a day. It takes strategy, investment, and team awareness. But when I see a report showing hundreds of personal data points removed before a document is published, I know we’ve taken a meaningful step forward.

I’ll end this personal saga with one clear belief: protecting personal data in documents is an ongoing mission, and it has to be automated. You just can’t hold back the flood of personal and sensitive data with human effort alone and a few black boxes. Whether it’s a carefully managed cloud AI model or a robust on-prem solution, AI is now a critical ally in this fight. With care, technical creativity, and a little humor to stay sane, it is possible to deliver high-quality public information without sacrificing individual privacy. And if I can help my team sleep better and cut down on coffee while doing it, even better.

Want a system that does this automatically? Click here.

Every redacted data point is one less headache down the road.