The Risks of Automating Social Security: Lessons from Recent AI Implementations

Exploring the challenges and failures of integrating AI into social security administrations, highlighting the importance of human oversight and the potential dangers of over-automation.

Let’s talk about how tech was supposed to make everything easier, but sometimes, it just doesn’t. Take the recent flop with automating social security—yeah, that didn’t go as planned. They rolled out this ChatGPT wannabe, the Agency Support Companion, to help out the staff. Sounds cool, right? Except it was anything but. The bot’s answers were about as helpful as a chocolate teapot—vague, often wrong, and just plain frustrating. And don’t get me started on the training video. It was like they forgot to include the part about, you know, actually keeping people safe. Not exactly a confidence booster.

And it’s not just a one-off. Brazil tried something similar with the Meu INSS app, aiming to streamline social security claims using fancy tech like computer vision. But here’s the kicker: it started rejecting legit claims over tiny mistakes. Imagine being denied because of a typo—now you’re stuck in legal limbo. It’s especially tough for folks in rural areas who might not be tech-savvy. So much for making life easier.

Now, there’s this whole ‘AI-first’ buzz in social services, pushed by folks who probably haven’t missed a paycheck in their lives. But here’s the thing: when you cut out human oversight, things go sideways. Like that time living benefit recipients were accidentally marked as deceased. Oops? More like a disaster. And the idea of rewriting an entire social security system’s codebase in months, maybe with AI’s help? That’s not ambitious—it’s borderline reckless.

All this makes you think: sure, tech can be a game-changer, but not when it’s rushed or done without a human touch. These examples are a wake-up call. Innovation’s great, but let’s not forget the people it’s supposed to help. The goal should be tech that supports, not replaces, the human element—because at the end of the day, that’s what really matters.