Back in July, I wrote Close the Gate, after yet another round of stories about AI agents “escaping”, “manipulating” people and finding ways around the restrictions humans had put in front of them.
My argument was fairly simple:
If you give an AI a goal, give it tools, and leave a route available that helps it achieve that goal, don't be terribly surprised when it uses it.
Or, in the terminology of the original post:
Check the gate before blaming the puppy.
Two months later, the puppy is back.
This time, I've been reading about AI agents finding vulnerabilities in IT systems. One line in particular caught my attention:
“AI has exploited issues in IT systems that humans simply haven’t got around to finding.”
And my immediate reaction was:
Hang on. Isn't that the gate again?
There is an important security issue here. If an AI system can access something it shouldn't be able to access, that's a problem. If an agent can take actions outside the boundaries its designers intended, that's a problem too.
But finding an unintended route towards an objective isn't necessarily evidence that the system has developed an intention to break the rules.
It might be evidence that we haven't actually specified the rules very well.
Humans are remarkably good at treating barriers as instructions.
A locked door doesn't just mean this particular method of opening the door has failed. It means you aren't supposed to be in there.
A login screen isn't merely an obstacle between us and some data. We understand the social, legal and organisational rules represented by it.
We know what the gate means.
An AI agent may simply know that the gate is closed.
And if we've told it to reach something on the other side, given it the ability to search for another route, and failed to explicitly or technically prevent those alternative routes, perhaps we shouldn't be astonished when it finds one.
This doesn't make the security problem less important.
Actually, I think it makes it more important.
Because “we need to make AI want to behave itself” is an enormously complicated problem.
“We need to build systems where prohibited actions are actually prohibited” is an engineering problem.
And engineering problems are things we know how to work on.
In July, I wrote that good systems don't rely on perfect behaviour. They use layers. You train the puppy not to run away.
But you still close the gate.
Now, apparently, the puppy has started finding holes in the fence.
Excellent.
Fix those too.