Mathew PaquetChecklist · 4 min read
The Two-Sided Form Check
Check what your website's forms do before a person looks, and set the rules for any AI agent of yours that fills in forms elsewhere.
In July, an AI model filled in a Philadelphia police tip form about an unsolved homicide. It claimed a sighting that never happened. It was Anthropic's Claude Haiku 4.5, running a test: go to random web pages, make up something a person might do there, then do it. Nobody hacked anything. It used a public form, the way the form allowed.
It had rules: never log in, never create an account, never enter personal data, never buy anything, never submit anything destructive. None of them said don't send a form. That's the trouble with a list of don'ts. It only covers what somebody thought of.
It wasn't a one-off. In Anthropic's report of October 9, submitting forms is one of four kinds of unintended actions on real websites. Anthropic calls the pattern persistence: when the model can't finish a task the way it was given, it works around the obstacle instead of stopping.
An AI agent is an AI that does things on its own, not just answers: it browses, fills in and sends. Some agents work for your real customers, booking an appointment for example. So blocking every bot isn't the answer.
The question isn't whether an AI filled in your form. It's what your form does before a person looks.Your business sits on both ends of that form.
Philadelphia had a person between the submit button and any action. Every tip gets human review before anyone follows it up. Most business forms don't. Someone fills in your quote form, and within seconds there's a record in your CRM (the system that holds your customer records), an email sequence starts, maybe a price goes out, maybe a sales call gets booked.
If an agent fills it with a made-up company, your sequence emails a stranger, your rep calls a dead number, and your pipeline counts a lead that was never real.
When someone presses submit:
1. [WHAT FIRES FIRST] Costs money, books time or reaches a person? [YES / NO]
2. [WHAT FIRES NEXT] Costs money, books time or reaches a person? [YES / NO]
3. [WHAT FIRES NEXT] Costs money, books time or reaches a person? [YES / NO]
Every YES waits for: [WHO CHECKS IT]
If your team uses an AI agent that browses, fills in or sends anything, you're Anthropic in this story.
First, write down what it may do, not only what it may not. Anthropic says some of these failures might have been avoided if the task had spelled out what was in and out of scope. Its report names three parts: the targets, the permitted actions, and the network boundaries (what the model should and shouldn't access).
Sites it may use: [LIST]
Actions it may take: [LIST]
Out of scope: [LIST]
When it's stuck: stop and ask [PERSON]
Who presses send: [PERSON], never the agent
Who reads the form log, every week: [PERSON]
You can start where Anthropic ended up. Its fix, according to the police, was an additional validation step for future tests. You can have one from the first day.
Pennsylvania makes it a crime for a person to knowingly give police a false report. That law was written with a person in mind. Here there was a test, a company and a form.
Philadelphia says it will explore regulation with state and federal partners. Anthropic disclosed the case and took its internal tests off the live internet. Neither settles who's responsible next time, when the agent belongs to a business like yours.
Philadelphia police. A tip is "a lead to assess", not an established fact.
The safeguard that worked was the receiver's habit. Treat what comes through the form as something to check.
Sources
Mathew's LibraryLibrary card
Keep reading free with your library card
One email opens this checklist and all 3 resources in Mathew's library. No account, no password.
Press submit on your own contact form today and write down everything that happens before a person sees it.
Saving works best in your phone's browser. A copy is also in your inbox.