The Pattern

Every week another CISO or CTO reaches out with a variation of the same situation: their organisation completed an AI transformation programme twelve to eighteen months ago. The consultancy was reputable. The governance documentation was thorough. The board presentation was compelling.

And now, as the first systems move into production, something is wrong.

The roadmap assumed a deterministic system. The actual AI system is not deterministic. The governance framework assumed defined inputs and outputs. The LLM produces neither. The security assessment covered the API layer. Nobody looked at what happens when the model’s instructions are rewritten mid-conversation.

This is not an edge case. This is the default outcome when AI strategy is decoupled from AI security.

What the Consultancies Missed

Traditional strategy consulting applies pattern-matching from previous technology waves — ERP implementations, cloud migrations, digital transformation programmes. These frameworks share a common assumption: the system does what it is told.

LLMs do not behave this way. A well-crafted input can override a system prompt. Training data leaks through inference. Agentic systems with tool access can be redirected mid-execution. None of these failure modes appear in an architecture review document.

The result is governance frameworks that are technically coherent and practically useless against the actual threat model.

The Remediation Path

Effective remediation starts from the attack surface outward — not from the governance document inward. The first question is not “does our governance framework cover this?” but “what can an attacker do to this system that we have not anticipated?”

That requires adversarial testing. Specifically, it requires automated adversarial tooling that probes LLM deployments the same way a real attacker would: through prompt injection, tool misuse, data exfiltration attempts, role confusion, retrieval poisoning, and attempts to push the system outside its intended operating boundaries.

Security-first remediation does not mean slowing the business down. It means rebuilding the AI programme around the parts that actually fail in production.

The New Operating Model

A useful AI remediation programme has four practical layers.

First, inventory the real systems. Not the slide-deck version. The actual models, APIs, retrieval pipelines, agents, plugins, prompts, datasets, logs, and human approval steps.

Second, test the system as an adversary would. Red-team the model, the retrieval layer, the workflow, and the surrounding application. A chatbot with access to documents is not just a chatbot. It is a new interface to internal knowledge.

Third, rebuild the controls around behaviour. Classical controls still matter, but they are not enough. AI systems need prompt and context hardening, tool-permission boundaries, output monitoring, abuse-case testing, logging, and clear escalation paths when the system behaves outside tolerance.

Fourth, measure residual risk honestly. The goal is not to declare the system safe. The goal is to know what can go wrong, how likely it is, how quickly it can be detected, and what the organisation will do when it happens.

What to Do Now

If your AI strategy engagement produced governance documents but no adversarial test results, start there. Pick one production or near-production AI workflow and ask five concrete questions:

These questions are uncomfortable because they move the conversation from intention to exposure. That is exactly why they are useful.

AI strategy failed in many organisations because it treated AI as another software rollout. It is not. It is a probabilistic system connected to business processes, data, and people. The remediation path is not another strategy deck. It is a security-led operating model that tests the system continuously and treats uncertainty as a design constraint.

That is where AI governance becomes real: not when the policy is written, but when the organisation can prove how the system behaves under pressure.