Causal Reasoning for Agentic AI

Why the harness — not the model — is now the bottleneck

An agent I watched last quarter did everything right and still got the answer wrong. It read the request, produced a clean five-step plan, called the right tools in the right order, and returned a confidently formatted result. The failure was two levels up: step three should never have run at all. The customer had failed a credit check in step two, and nothing in the plan encoded the idea that failing a credit check causes the rest of the sequence to stop mattering.

No amount of retry logic fixes that. The execution was fine. The plan was wrong.

This is the part of the agentic stack that the current conversation about harnesses tends to skip over, and it is the part I want to spend this article on.

1. The harness solved the wrong half of the problem

Harness is the term of the moment. The argument behind it is correct and worth restating: building an agent is now easy, and running agents at enterprise scale, consistently, is where the work actually is. The centre of gravity has moved from picking a model to building the machinery around it — so that execution comes with reliability and accountability guarantees.

Anthropic popularised the framing for coding agents, but it generalises to any agentic design. The building blocks look roughly like this:

• Context and memory management — compress context, summarise into short-term memory, promote what matters into long-term memory.
• Reasoning loop — generate the plan, execute it, reflect, adapt.
• Evals — LLM-as-a-judge over responses and outcomes, anchored to ground truth where ground truth exists.
• Security — standardised, scalable paths from user to application to agent to tool to source system.
• Guardrails — prompt injection, tool misuse, misaligned actions, memory poisoning.
• Human-in-the-loop — humans as participants in the lifecycle, not reviewers bolted on at the end.

Five of those six manage execution. Only the first one touches the plan itself, and that first block is still, in almost every production system I have seen, a single call to an LLM asking it to decompose a task.

Which means agentic systems inherit, wholesale, whatever the reasoning limits of that model happen to be.

2. Every real workflow branches, and the branches are causal

Take a mundane e-commerce order.

Place order → check credit → reserve stock → choose delivery mode. Two of those nodes are not deterministic. Check credit can go either way. Delivery mode forks into store pickup or shipment. Shipping is a task that may simply never execute in a given run.

A plan generated as a linear sequence quietly assumes every node fires. And the relationships that determine whether a node fires are not statistical. “Credit declined” does not correlate with “skip fulfilment” — it causes it. Getting that branch right requires a model of consequence, not a model of co-occurrence.

The cost compounds. A step that is 95% correct is about 60% correct over ten steps. Long-horizon agents rarely fail inside a tool call; they fail at the joins.

So the requirement is: multi-step reasoning that understands the causal relationships between component agents. Which brings us to what the research says about whether today’s models can do that.

3. What the reasoning research actually shows — and what it doesn’t

Two Apple papers get cited constantly here, usually more confidently than they deserve. Both are worth reading; so is the rebuttal.

Findings and rebuttal, side by side.

GSM-Symbolic (2024) found that state-of-the-art models degrade when you vary the surface of a maths problem: swap names and numbers and scores move; add a clause that is topical but irrelevant and scores fall sharply — reported at up to a 65% drop on the hardest variant. Same problem, different clothes, different answer.

The Illusion of Thinking (2025) used controlled puzzle environments with tunable complexity — Tower of Hanoi, river crossing, blocks world — and reported a three-zone pattern: plain LLMs ahead on easy instances, reasoning models ahead in the middle band, both collapsing to near-zero past a complexity threshold. It also reported a counter-intuitive scaling behaviour: reasoning effort rose with difficulty, peaked, then declined despite an adequate token budget.

Then the pushback. In The Illusion of the Illusion of Thinking (2025), Alex Lawsen and Claude Opus argued that a chunk of the reported collapse was an artefact of the setup rather than a limit of reasoning. Several failure points sat exactly where models hit output-token ceilings — a 15-disk Hanoi solution runs to over 32,000 moves, and models were explicitly saying they were stopping to save length. Some river-crossing instances were mathematically unsolvable, and models were penalised for saying so. And when asked to emit a generating function instead of an exhaustive move list, models solved instances far beyond the reported collapse point.

The rebuttal is not the last word either — replications have softened parts of the original claim while upholding the core observation that these models generalise poorly past a certain complexity. Lawsen himself was surprised how seriously the paper got taken.

So here is the honest read, which is weaker than the version that usually gets quoted:

It is not established that LLMs cannot reason. It is established that measured reasoning degrades unpredictably with restatement, distractors, and horizon length — and that we cannot currently distinguish a capability limit from a harness limit by inspection.

For anyone designing autonomous execution inside an enterprise, that weaker claim has the same engineering consequence as the strong one. An unpredictable degradation curve is as unusable as a hard ceiling. You cannot write an SLA against “usually fine, unless the request is phrased unusually.”

The design response is the same either way: stop leaving plan quality to depend on phrasing. Structure the reasoning externally.

4. Correlation and causation, as an engineering distinction

The familiar line — correlation doesn’t imply causation — is usually deployed philosophically. It is more useful as a statement about which artefacts a system can produce.

Correlation-based vs causal-informed planning.

A correlation-based planner ranks next steps by resemblance to things it has seen. It reorders steps when phrasing changes. It cannot separate a driver from a confounder. It explains itself by narrating after the fact. Ask it a what-if, and it answers by imagination. When it fails, it fails opaquely — the plan still reads beautifully.

A causally-informed planner ranks next steps by their effect on the target outcome. It holds under restatement, because it is keyed to structure rather than surface. It names confounders explicitly. It explains itself from the graph it acted on. Ask it a what-if, and it answers by simulating an intervention. When it fails, a missing edge is a visible gap.

Gartner put causal AI at the Innovation Trigger stage of its 2024 AI Hype Cycle for roughly this reason: moving beyond correlation-based prediction toward systems that can prescribe actions and act more autonomously.

The practical caveat, which the enthusiasm usually skips: causal structure is expensive. Correlation is cheap and abundant. So spend the expensive thing where a wrong branch is irreversible — payments, credit, safety, disclosure — and let correlation carry the rest.

5. Design the agents from the workflow, with humans inside the loop

If workflows are causal, the design process should start from the workflow — and should keep humans in it at more than one point.

Today, human involvement in the agentic lifecycle is mostly oversight, which mostly exists to satisfy legal and compliance. That is a waste of the most informed participant in the system. There are four points where intervention actually improves the next cycle:

The four human intervention points.

• Co-plan — validate that the generated orchestration graph matches the stated intent, before any tool is invoked.
• Co-execute — pause mid-run and correct course when a response drifts, or when the long-horizon goal is clearly out of reach.
• Co-comply — mark the irreversible tasks (payments, filings, customer commitments) and gate them behind policy checks.
• Co-memorise — curate what enters long-term memory. Unreviewed memory is where yesterday’s error becomes tomorrow’s prior.

As with state checkpointing, the frequency and format of the feedback matter as much as the fact of it. Log every intervention as structured evidence, and use the accumulated history to decide where the next intervention is worth a human’s time.

Co-planning from the SOP you already own

For the co-plan step specifically, the best starting artefact is usually one the enterprise already has: the standard operating procedure. An SOP is a causal document. It encodes what must happen before what, and why.

• Ingest the process knowledge — SOPs, process maps, control narratives.
• Extract candidate steps with an LLM and map them to agent functions.
• Ground each step with a citation back to its source paragraph, so hallucinated steps are visible as steps with no source.
• Decompose complex steps into atomic tasks, especially where conditions are joined by a logical OR, and merge duplicates.
• Exemplify — generate a compliant example and a violating example per step. These resolve ambiguity before implementation, and they double as eval cases later.

Two gates matter after that. A conversational review lets subject-matter experts redesign the generated steps — this is what stops you from agentifying a bad manual process, which is the single most common way these programmes waste a year. And the validated process gets matched against registered agents and tools in a marketplace, so reuse is the default and compliance is inherited rather than reimplemented.

6. Causality in training: supplying the WHY

Pre-training on text answers what usually follows. Knowledge graphs add how things relate. Causal models add the ingredient neither provides: why one thing moves another.

Concretely, what a causal component model can hand a planner:

Five causal ingredients.

• Root cause — detect and rank the drivers behind an outcome, by strength of effect.
• What-if and counterfactual — estimate the consequence of an alternative action against the factual state.
• Explainability — state why one action beats another, from structure rather than narration.
• Confounders — surface hidden or misleading influences before they enter the plan.
• Pathways — sequence interrelated actions along the route that actually moves the outcome.

These get built as causal component models: the system is split into components that each model one set of cause-and-effect relationships, then interconnected. Components ship independently, get versioned, and get replaced without retraining the whole. Over time, reinforcement learning and self-taught reasoning methods close the loop, with models and agents learning from each other’s actions, rewards and policies through managed transfer.

Here is what that looks like on a real decision.

A causal model of credit risk.

Debt-to-income ratio and credit utilisation push creditworthiness down; payment history and employment tenure push it up. Edge weight is the estimated strength of the causal effect, not a correlation coefficient. The agent evaluating “raise this limit or not?” can simulate the intervention rather than pattern-match to similar applicants — and then hand the reviewer the same graph it reasoned over.

The output stops being a score. It becomes a score, the drivers ranked by influence, and the counterfactual that would have changed the decision.

The caveat this argument needs. Causal graphs are not free, and they are not discovered. They require domain assumptions, and observational data alone rarely identifies a unique structure — several different graphs can fit the same data equally well. Treat the graph as a reviewed artefact with an owner, not as ground truth that fell out of a pipeline. A confidently wrong causal model is worse than an honestly uncertain correlational one, because people trust it more.

This is also the strongest practical argument for domain-specific small language models. They are narrow enough to be grounded in a causal structure someone has actually verified, and cheap enough to run per workflow.

7. Causality at inference: adding introspection to ReAct

Retraining is a long lever. There is a shorter one available today, and it operates entirely at inference.

Standard ReAct agents are effective for web retrieval. In industrial and IoT environments, they are not, and the failures are specific:

• Domain linkage missed. Chiller unit tonnage and energy efficiency are causally linked — a vital relationship in this environment — and the agent treated them as unrelated fields.
• Date-offset drift. “Last week” was resolved inconsistently across runs, silently changing the answer.
• Premature termination. Multi-step composition abandoned mid-chain, with redundant tool calls before the stop.

The fix wraps ReAct rather than replacing it:

• A distillation module pre-processes the query into structured semantic units: variables, constraints, goals.
• ReAct still generates the execution plan — but before executing, the orchestrator emits an internal disambiguation sub-query and answers it. This anticipatory step is cheap, and it measurably improves plan coherence, task adherence, and tool-invocation accuracy.
• A review module acts as an LLM-as-a-judge, classifying the final output as accomplished, partially accomplished, or failed against the original query.
• A reflect module then performs introspection over the execution plan — assessing the reasoning steps and tool invocations — and emits targeted feedback: plan shifts and reasoning templates appended to the system prompt for future runs.

The reflection prompt is deliberately blunt about the premise that something went wrong:

You are an advanced reasoning agent that can improve based on introspection.
You will be given a previous reasoning trial in which you had access to
multiple agents and tools, and a query to answer.

You were unsuccessful in resolving the query — either because you
misunderstood it, or because you used up your allotted reasoning steps.

In a few sentences, diagnose a possible reason for the failure and devise a
new high-level execution plan that mitigates the same failure. Use complete
sentences.

Examples:
{examples}

Previous trial:
Query: {query}
{plan}

What comes back is not a retry. It is a diagnosis in plain sentences plus a revised high-level plan, stored as a reusable template instead of being discarded with the session.

This is the cheapest causal intervention available right now: no retraining, inference-time only, and it leaves an auditable trail.

One discipline to go with it: report the delta, not the vibe. Track plan coherence, tool-invocation accuracy, and task-completion rate per query class, before and after. “It feels better” is not a result.

8. Where to start on Monday

None of this requires a frontier training run. Four of these five steps are harness work you can do this quarter.

• Instrument the plan. Log every generated plan, branch taken, and tool call as structured evidence. You cannot improve a loop you cannot replay.
• Add introspection. Wrap the existing ReAct agent with distil, judge, and reflect. Measure it against the logged baseline.
• Map one workflow causally. Pick a single high-consequence decision. Draw the graph with your SMEs. Review it as an artefact — do not discover it and ship it.
• Gate the irreversible. Route payments, credit and disclosure through co-comply. Everything else runs unattended and gets sampled.
• Promote what survives. Reflection templates and reviewed graphs graduate into long-term memory. Nothing enters unreviewed.

Closing

The reasoning limits of current models ripple straight into enterprise adoption of agentic AI, because those models are still doing the planning. The harness manages what happens after the plan; it does not repair the plan.

Business workflows are causal by construction, and the causal ingredient is the one that is missing — in training, where knowledge graphs and causal component models can supply it, and at inference, where introspection can approximate it today for the price of a few extra tokens.

I would not overclaim the destination. Causal discovery is hard, the graphs need owners, and the research on model reasoning is genuinely contested. But the direction seems defensible: as enterprise processes keep changing, the systems that can explain why an action worked will outlive the ones that can only predict what usually comes next.

References

• Mirzadeh et al., GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (Apple, 2024)
• Shojaee et al., The Illusion of Thinking (Apple, 2025)
• Lawsen & Opus, The Illusion of the Illusion of Thinking (arXiv, 2025)
• Gartner, Hype Cycle for Artificial Intelligence, 2024

Originally published on Medium.