Last week, West Midlands Police got itself into a mess after admitting that a piece of “intelligence” used in planning for an Aston Villa match contained something that simply was not true, something that AI had just made up. A report referenced a West Ham vs Maccabi Tel Aviv match that never happened; they had never played this, yet Copilot said it did. The Chief Constable, after denying it, later said the error came from the Police's use of Microsoft Copilot, not from a human source, and the claim was used without proper checking, a rookie error for the Keystone Cops
The story has had real consequences. After sustained criticism of the force’s handling of the decision, West Midlands Police Chief Constable Craig Guildford has now stepped down with immediate effect. It is a reminder that AI mistakes are not “tech problems”, they are human ones. They become leadership problems the moment they influence real decisions.
That single mistake became part of a much bigger controversy about banning Maccabi Tel Aviv fans from attending the Europa League fixture at Villa Park on 6 November 2025, and it has now contributed to leadership fallout at the force.
So let's explore this a bit further and see what went wrong and what you can do to prevent this from happening to you.
A hallucination is when an AI system produces information that looks confident and plausible, but is wrong or made up.
Not “a typo”. Not “a slightly dated fact”. Fully invented details that fit the pattern of what you asked for. AI can do this all the time; with normal prompting, you get this about 15% of the time.
That is exactly what happened here: Copilot produced a credible-sounding reference (a football match in this case) that did not exist, and it was allowed to travel into a real-world decision without being verified, which had human impacts. Imagine if you were making a decision about dodgy information, how confident would you be?
Most AI tools are not databases. They are pattern machines.
They generate the most likely next words based on training data and context, and if you ask them something where:
The answer is unclear
The sources are mixed
The tool cannot access the right documents
or the prompt encourages “fill in the gaps”
…they will sometimes guess.
So if you wrote Mary had a little......... what would the next word be, well you might say lamb but equally as correct could be sleep, brother or drink, all of these fit the sentence, and that's what AI does, it tries to guess the next likely word.
That is not a moral failing. It is a design trade-off, and it can be useful.
Here is the part most people miss: hallucinations are a side effect of the same capability that makes AI valuable.
If a model could only repeat what it is 100% certain about, it would be a dull search box.
The value of modern AI is that it can:
draft, summarise, structure and reframe fast
generate options when you are stuck
generate and surface angles to questions you might not have considered
help you think, not just look things up
The trick is simple: use it for thinking and drafting, not as your source of truth.
You asked for practical steps that make readers think: “I need training on this.” Good. Here are the ones that matter, without the tech fluff.
Use prompts like:
“List the claims you are making that would need verification.”
“What are you uncertain about?”
“What would change your answer?”
“Give me three alternative explanations.”
This does not guarantee truth, but it forces the model out of autopilot and makes risk visible.
If the tool cannot provide links or named sources, assume it might be guessing.
Even when it does provide citations, spot-check them. (A link is not the same as evidence.)
This is the easiest behaviour change for busy leaders: no sources, no trust.
RAG (retrieval augmented generation) simply means: give the AI the documents you want it to rely on.
Instead of asking “what is our policy?”, you feed it the policy. Instead of “what did the contract say?”, you provide the contract.
You are not making the model smarter. You are making it grounded.
If the output could affect:
safety
reputation
legal position
finance
hiring and people issues
…then you need a human sign-off and a simple verification step.
West Midlands Police did not get in trouble because they used AI. They got in trouble because a made up claim was allowed to behave like intelligence.
A practical minimum:
What did we ask?
What did it say?
What did we check?
What did we change?
Who approved it?
That is governance without bureaucracy.
Do not panic about hallucinations. Panic is a waste of time.
Do two things instead:
Decide what AI is allowed to do in your business (draft, summarise, brainstorm, analyse)
Decide what AI is never allowed to do without checking (facts, claims, policy, compliance, anything high impact)
If you do that, hallucinations become manageable. They stop being scary and start being just another risk to control, like any tool.
This is exactly the gap we see in SMEs.
Leaders buy Copilot or ChatGPT, people start using it, then everyone quietly hopes it will be fine. No training, no rules, no verification habits, no RAG, no process, and we end up with errors.
If you want to fix that properly, our training focuses on:
How to prompt for accuracy (and spot uncertainty)
How to build “check itself” habits into everyday use
When and how to use RAG so answers are grounded in your documents
What lightweight governance looks like for a small team
Because the goal is not “more AI”.
The goal is better decisions, with fewer avoidable mistakes.
If this story made you think “we need something comprehensive to stop this happening here”, our ConkerAI training courses are designed for exactly that.