Every week, another enterprise announces a successful AI agent pilot. And then — nothing. The pilot runs for three months, the team presents impressive results in a controlled environment, and then the project quietly stalls before it reaches production. This pattern has become so common that it has its own name in analyst circles: pilot purgatory.

The question worth asking is not why some AI agents fail. It's why almost all of them fail to scale beyond the initial proof of concept — and what the exceptions have in common.

The capability illusion

Modern large language models are genuinely impressive. They can summarise complex documents, answer questions across a wide range of domains, write code, analyse data, and engage in sophisticated reasoning. When you show a business leader a demo of an AI agent handling their reconciliation workflow or processing regulatory documents, they see exactly what they hoped to see.

The problem is that the demo is lying — not intentionally, but structurally. Demo environments are controlled. The data is clean. The edge cases have been quietly removed. The system prompt has been carefully tuned. And when a general-purpose model is prompted well, in a controlled setting, with clean inputs, it can appear to handle almost anything.

"Production is not a controlled environment. Production is where the edge cases live, where the data is messy, where the regulatory requirements are specific, and where a failure has real consequences."

This is the capability illusion: confusing what a general-purpose model can do in a demo with what a deployed agent can do reliably, every day, in your actual operational environment.

What actually breaks

When AI agent deployments fail in production, they fail in predictable ways. Understanding these failure modes is the first step to avoiding them.

Domain knowledge gaps

A general-purpose model trained on broad internet data does not know your specific reconciliation logic. It doesn't know that your institution uses a non-standard field mapping in your SWIFT messages, or that your exception threshold for currency mismatches is 0.005% rather than the industry standard 0.01%, or that your regulatory reporting requires a specific audit trail format mandated by your local regulator rather than the generic PSD2 template.

These details seem minor in isolation. In production, they're the difference between an agent that works and one that generates exceptions on 40% of transactions.

System integration failures

Real enterprise workflows don't live in a single clean API. They span legacy systems with inconsistent field naming, ERPs with bespoke data models, document stores with varied formats, and real-time feeds with unpredictable latency. A general-purpose agent that hasn't been trained on these specific integration patterns will fail — not catastrophically, but gradually, as edge cases accumulate.

Regulatory brittleness

Regulated environments are unforgiving. An agent that generates a report with the wrong data lineage, or misapplies an exception rule, or produces output that fails a governance check, creates a compliance problem — not just a technical one. General-purpose models don't have the embedded regulatory knowledge to navigate these requirements reliably.

What vertical training changes

The agents that successfully reach production — and stay there — share one characteristic: they were trained on the specific domain they operate in. Not fine-tuned on a handful of examples, but genuinely built with domain knowledge as a foundational design decision.

What does this mean in practice? It means the agent's knowledge base includes your specific workflow logic, not generic process templates. It means the integration layer was built for your actual systems, not generic connectors. It means the exception handling reflects your actual regulatory obligations, not industry averages.

The training investment pays off differently

Teams often resist vertical training because it feels expensive relative to prompting a general model. This comparison misunderstands where the cost actually lies. The cost of a failed production deployment — the engineering time, the remediation work, the lost credibility with the business, the risk of a compliance incident — is orders of magnitude higher than the cost of building domain knowledge into the agent from the start.

A different way to think about agents

The most useful reframe is this: stop thinking about AI agents as AI products, and start thinking about them as operational systems that happen to use AI as the reasoning layer. Operational systems need to be reliable, auditable, and fit for the specific environment they run in. They need to handle edge cases, not just common cases. They need to fail gracefully, not silently.

When you evaluate an AI agent through this lens — as an operational system — the requirements become clear. General-purpose is not fit for purpose. Vertical training is not optional. And demo success is not a reliable predictor of production success.

The agents that are working in production today — genuinely working, at scale, in regulated environments — were all built this way. The pattern is consistent enough that it's no longer a hypothesis. It's a precondition.

APIDNA builds vertically-trained agentic systems for enterprise environments. If you're evaluating AI agents for operational workflows, get in touch.

The AI industry has done an excellent job of making general-purpose models look like domain specialists. When you ask a frontier model a question about financial reconciliation, it will give you a confident, coherent answer. When you ask it to help structure a compliance workflow, it will produce something that looks reasonable. The surface quality is high enough that it creates a convincing impression of domain expertise.

This impression is useful for understanding what AI can do in principle. It is not useful for predicting whether an AI agent will work in your production environment.

What "general-purpose" actually means

A general-purpose language model is trained to predict useful text across a very wide distribution of inputs. It has seen examples of financial workflows, regulatory documents, and operational processes — but as text, not as executable knowledge. It knows what reconciliation looks like in general. It does not know what reconciliation looks like in your institution, with your data model, your exception logic, your regulatory obligations, and your system architecture.

This is not a limitation of the model — it's a design characteristic. General-purpose models are optimised for breadth. Enterprise operations require depth.

"A general-purpose model knows that reconciliation happens. A vertical agent knows how your reconciliation happens — and that's an entirely different kind of knowledge."

The four dimensions where vertical training matters

1. Workflow logic

Every organisation has operational workflows that diverge from industry templates in ways that matter. Your payment matching tolerance is different. Your exception routing follows a specific escalation matrix. Your regulatory reporting has a format mandated by your regulator, not the industry standard. A general-purpose agent cannot know these specifics without being explicitly trained on them. And when it guesses — which it will — those guesses will be wrong in ways that are difficult to detect until they cause a problem.

2. Data structures and schemas

Enterprise data is not clean. It lives across multiple systems with inconsistent field naming, custom extensions to standard schemas, legacy data formats, and idiosyncratic edge cases accumulated over decades of operational history. A vertically-trained agent has been built with this complexity in mind. A general-purpose agent encounters it and produces unpredictable results.

3. Regulatory context

Compliance requirements are not general — they are specific to jurisdiction, institution type, and regulatory regime. The PSD2 reporting requirements for a UK payment institution differ from those for a continental European bank. DORA obligations vary by entity classification. A vertically-trained agent encodes the specific regulatory context for your institution. A general-purpose agent applies generic regulatory knowledge, which is accurate enough to sound correct and wrong enough to create compliance risk.

4. Exception handling

In operational workflows, the common case is not the hard case. The hard cases — the exceptions, the edge cases, the anomalies — are where the value of domain knowledge becomes most apparent. A vertically-trained agent has been built with your exception logic embedded. It knows what to do when a transaction matches on three of four fields but not the fourth. A general-purpose agent improvises, and improvisation in a compliance-sensitive workflow is precisely what you don't want.

The practical test

There is a straightforward way to evaluate whether the distinction matters for your use case. Ask these questions about the agent you're evaluating:

• Can it handle the specific edge cases in your workflow — not generic edge cases, but the ones your team encounters every week?
• Does it understand your data model — the specific fields, formats, and relationships in your systems?
• Can it apply your regulatory obligations correctly — not the general regulatory framework, but the specific requirements for your institution and jurisdiction?
• When it fails, does it fail gracefully and escalate correctly — or does it fail silently or produce plausible-looking incorrect output?

If the answers to these questions are yes, you have a vertically-trained agent. If the answers are "it can handle most cases" or "it works well for standard scenarios", you have a general-purpose model that will struggle in production.

The organisations that have learned this lesson the hard way tend to learn it the same way: a general-purpose agent that worked well in a pilot environment fails in ways that are difficult to diagnose, and the root cause is always the same. The model didn't know enough about the specific domain to be reliable.

The organisations that have learned it the easy way built domain specificity into the agent design from the start. They took longer to get to the first demo. They got to production faster, and stayed there.

APIDNA builds vertically-trained agentic systems for enterprise environments. Talk to us about your workflows.

When people talk about AI agents, they typically focus on the model. Which LLM? How large? How accurate on benchmarks? These are reasonable questions, and the answers matter. But they are not the questions that determine whether an AI agent works in a production enterprise environment.

The question that determines production viability is: what is the agent running on? The model is the reasoning layer. What sits beneath it — the infrastructure that connects the model to your systems, manages workflow state, handles failures, logs decisions, and surfaces exceptions to the right humans — is the execution layer. And it's the part that most AI agent deployments get wrong.

Why the model is not enough

Consider a financial reconciliation workflow. The agent needs to pull transaction data from multiple systems, apply matching logic, identify exceptions, route those exceptions appropriately, generate a reconciliation report, and log every decision in a format that satisfies your audit requirements. All of this needs to happen reliably, every day, with correct handling of the edge cases that appear regularly in real transaction data.

The model handles the reasoning steps — understanding the matching logic, identifying anomalies, deciding how to categorise an exception. But reasoning is only one part of what the workflow requires. The rest — connecting to the source systems, maintaining state across a multi-step process, handling failures in any individual step, producing output in the required format, logging the audit trail — that's the execution layer.

"You can have the best model in the world sitting on top of a weak execution layer, and you will have an agent that fails in production. The execution layer is load-bearing."

The five components of a production execution layer

1. System integration

Enterprise workflows span multiple systems — ERPs, databases, payment platforms, regulatory reporting systems, document stores, real-time feeds. Production-grade system integration means more than calling an API. It means handling authentication correctly, managing rate limits, dealing with schema variations across systems, handling partial failures gracefully, and managing the latency characteristics of each system without blocking the overall workflow.

Most AI agent frameworks treat integration as an afterthought — a list of tools the agent can call. Production integration is an engineering discipline in its own right.

2. Workflow state management

Multi-step workflows need persistent state. If a reconciliation job processes 2.4 million transactions over several hours and the system encounters a transient failure at step 3 of 8, it should resume from step 3 — not restart from the beginning. If a compliance report generation is interrupted, the state of the partially-completed report needs to be maintained so the process can continue correctly.

State management is often invisible until it fails. When it fails in a complex workflow, the consequences are difficult to diagnose and expensive to remediate.

3. Human escalation paths

The most important design decision in any production agent deployment is: what happens when the agent encounters something it cannot handle correctly? For regulated workflows, the answer must always be: the right human is notified, with the right context, through the right channel.

This means defining — before deployment — exactly what constitutes an escalation-worthy exception, who receives it, in what format, with what context, and what the expected response time is. Agents that don't have well-defined escalation paths fail silently. Silent failure in a compliance-sensitive workflow is not acceptable.

4. Audit and observability

In regulated environments, the question is never just "did the agent produce the right output?" It's "can you prove that the agent produced the right output, using the correct methodology, based on the correct inputs, with appropriate human oversight at the right decision points?" This requires a level of observability that goes well beyond standard application logging.

5. Reliability and resilience

Production workflows run continuously. They encounter transient failures, network timeouts, system outages, and unexpected data. A production execution layer handles these gracefully — retrying where appropriate, failing safely where not, maintaining data integrity throughout. An agent that produces incorrect output because a downstream system was temporarily slow is not a reliable agent.

What this means for how you evaluate AI agents

When you're evaluating AI agents for enterprise workflows, the model capabilities are necessary but not sufficient. The questions you should be asking about the execution layer are:

• How does the agent connect to our existing systems — and how does it handle failures in those connections?
• How is workflow state maintained — what happens if the process is interrupted mid-execution?
• What are the escalation paths — how does the system decide when to involve a human, and what does that process look like?
• What is the audit trail — can we produce a full decision log that satisfies our regulatory and internal governance requirements?
• How does the system handle edge cases — not the common cases, but the exceptions that appear in real operational data?

Vendors who can answer these questions precisely, with specific implementation details rather than general principles, are building for production. Vendors who deflect to model capabilities or benchmark performance are building for demos.

The infrastructure investment

Building a production-grade execution layer is significant engineering work. It is also the work that most organisations underestimate when they plan AI agent deployments. The model selection takes a week. The execution layer takes months.

This is why many AI agent deployments stall between pilot and production. The pilot runs on a simplified version of the execution layer — carefully controlled inputs, manual handling of edge cases, human oversight of every decision. Scaling that to production requires building the infrastructure that makes the agent genuinely reliable. And that infrastructure is the hard part.

The organisations that have moved agents from pilot to production at scale have all invested in the execution layer first. The model is important. The execution layer is what makes it work.

APIDNA's platform is built execution-layer first — designed for the complexity of real enterprise deployments. Talk to us about how we approach this for your workflows.