From Coordination to Authority

Evidence Boundaries in Multi-Agent AI Reliability

Nicholas Zinner·Beacon Bot·August 9, 2026

Abstract

“Reliability” in LLM-based multi-agent systems now names several different research problems: harness sensitivity, coordination, failure attribution, error propagation, verification, permission, and provenance. Evidence about one is routinely used to support a claim about the next: communication is read as coordination, a critic as independent validation, consensus as permission, and a transcript as an accountable record.

We systematize a bounded 2025–2026 literature around those invalid transitions. The field’s sharpest disagreement, whether redundancy buys reliability, turns on architecture: independent parallel sampling with voting and role-differentiated pipelines with shared context have opposite error dynamics. Measurement creates a second fault line because mean improvement and tail corruption can coexist in the same cascade.

This is a source-grounded narrative systematization, not an exhaustive review or statistical meta-analysis.

01 / Problem

Reliability became several problems

Hold a model’s weights fixed and change only the scaffolding around it, including the prompt format, action grammar, retry policy, and orchestration loop, and measured agent performance moves by roughly ten to twenty percentage points on a fixed SWE-bench Verified subset.[2] The difference comes from details the benchmark did not report.

Consider a common system. A planner decomposes a task and assigns roles. Several agents exchange messages and produce an artifact. A verifier approves it against criteria the planner generated. A vote authorizes deployment, an executor changes external state, and the retained transcript does not establish which evidence was current, who possessed authority, or whether the resulting state matched the approved artifact.

Every component may have functioned exactly as implemented. The system can still fail because each stage answers a different question: was an artifact produced, did the agents preserve enough state to coordinate, was it correct, was the check independent, was the action permitted, and can another party reconstruct what happened? One success score cannot answer all six.

Figure 1 — An evidence chain, not a maturity ladder
Each transition requires separate evidence

Different methods establish different transitions. Success in one box does not establish the next. Durable reconstruction cuts across the chain and asks what remains inspectable afterward.

02 / State of the art

How the field got here

Failure taxonomy: making breakdowns legible

MAST organized observable breakdowns into fourteen failure modes across system design, inter-agent misalignment, and task verification.[1] Six experts developed the taxonomy from 150 traces; most of the roughly 1,600-trace corpus was annotated by an automated pipeline. Agreement on applying the refined taxonomy establishes consistent use, not that the categories are exhaustive, mutually exclusive, or uniquely causal.

Harness effects: changing the unit under test

Holding weights fixed while varying scaffolding moves scores by roughly 10–20 points on a fixed coding subset.[2] Harness-Bench reports 5,194 trajectories across eight model backends and six configurable harnesses while preserving traces, artifacts, validator outputs, token counts, permission violations, and process-quality signals.[3]

The execution condition is a model–harness–tool–environment configuration, including routing, permissions, budgets, timeouts, and fallback behavior. Complete-harness comparisons reveal sensitivity but do not isolate the mechanism behind each effect.

Communication is not coordination

Silo-Bench isolates a communication–reasoning gap: agents exchange messages successfully while failing to reason over information distributed across participants.[4] Coordination needs a usable representation of current ownership, dependencies, evidence, blockers, commitments, corrections, and completion criteria. Raw conversation may contain those facts without making them current, salient, or binding.

Attribution: from category to component and step

TraceElephant’s best static configurations reach roughly 65.9% attribution accuracy at agent level but only 30.3% at step level. Removing task inputs degrades step-level accuracy by up to 76% relative to full information.[5] MP-Bench shows that several causal perspectives can remain defensible for one trajectory,[6] while VerifyMAS tests whether a hypothesized error is present, explanatory, and attributable to a component or step.[7]

Fault injection and verification

MAS-FIRE introduces known disturbances and measures propagation, detection, silent failure, defensive response, and recovery, with recovery varying by topology.[9] Controlled intervention strengthens evidence about the tested mechanism without establishing production incidence.

VERIMAP moves checks into decomposition so intermediate outputs can be blocked before downstream work inherits them.[10] A deterministic oracle, hidden test, learned evaluator, same-context critic, human review, and formal proof still establish different things.

Authority enters the technical literature

Overlaying Governance models principals, recursive delegation, contextual scope, attenuation, expiry, and revocation as a compositional authorization layer.[11] Formal permission correctness shows that a system enforced a policy. It does not show that the decision was sound, adequately supervised, or legitimate.

03 / Disagreement

When more agents help, and when they do not

Two of the literature’s strongest recent results point in opposite directions.

Does redundancy buy reliability?

The Six Sigma Agent executes atomic, independently verifiable actions repeatedly across diverse LLMs, clusters the outputs, and takes the winning cluster. It reports 3.4 defects per million opportunities at thirteen agents. Its headline 14,700× improvement is measured against single GPT-4o-mini execution at 50,000 DPMO, the weakest baseline evaluated; the reported 80% cost reduction is relative to more expensive models, with roughly 47% added latency at five samples.[12]

By contrast, From Spark to Fire models collaboration as a dependency graph in which minor inaccuracies are reused until they harden into false consensus. A single injected error seed produces widespread failure; its governance layer prevents final infection in at least 89% of runs.[8]

The papers describe different systems. Six Sigma assumes statistically independent errors, per-action error below one half, and failures that do not converge on the same wrong answer. From Spark to Fire studies agents that consume one another’s outputs, where shared context is the collaboration mechanism.

Figure 2 — Two architectures sharing one name

Independence makes errors filterable; voting or verification performs the filtering. Shared-context pipelines can correlate errors by construction.

Independence is necessary but not sufficient. It makes errors distinguishable; voting or verification removes them. N-version programming found decades ago that independently written programs fail on correlated inputs more often than independence predicts.[18] Shared specifications created common-mode failures there; LLM pipelines add a direct inheritance channel through generated context.

The same cascade, measured two ways

Across 500 sequential experiments, ten domains, three heterogeneous models, and 1,250 responses, Hallucination Cascade reports normalized hallucination falling from 0.422 at the first agent to 0.272 at the final agent. Factual accuracy also falls, from 0.789 to 0.769.[16]

Our interpretation is that the cascade may be converging on claims that are harder to mark wrong rather than adding information. The accuracy decline is too small to establish that mechanism by itself; the divergence between metrics is the evidence.

The studies estimate different statistics. From Spark to Fire asks whether an injected error reaches the final artifact, a tail question. Hallucination Cascade measures average inconsistency. Smoothing and hedging may improve the average without protecting against one inherited false premise.

Figure 3 — Metric divergence in Hallucination Cascade

Only reported first-agent and final-agent endpoints are shown. No intermediate values are inferred.

When does adding agents help at all?

Across 260 configurations, performance relative to a single-agent baseline ranges from +80.8% on decomposable financial reasoning to −70.0% on sequential planning. Trace-level errors amplify 17.2× under independent parallel execution, 7.8× under decentralized coordination, and 4.4× under centralized coordination.[13] Here “independent” means parallel outputs aggregated without voting or verification. Independence makes errors filterable, but only aggregation or verification filters them.

Across seven benchmarks, a single agent role-playing a homogeneous workflow matches multi-agent performance while reusing its KV cache. OneFlow cuts GPT-4o mini inference cost from $0.198 to $0.026 on HumanEval and from $2.343 to $0.819 on MATH.[14]

A weaker vendor-authored study reports silent degradation increasing with chain length.[17] Its proprietary evidence and exponential “entropy principle” do not support a general law; the observation only reinforces the need to test depth explicitly.

Coordination helps when work is decomposable and verification is centralized. It hurts when work is sequential, context is shared, and no independent check sits between stages. Whether errors can correlate is the moderator, yet almost every paper in this corpus leaves it implicit.

A second, quieter dispute

Benchmarks that assign one responsible agent and step assume a privileged cause exists. MP-Bench shows that several attributions can remain defensible for one trajectory.[6] If failures are multi-causal, single-label ground truth rewards models for resolving ambiguity confidently rather than correctly.

04 / Evidence boundaries

Invalid shortcuts in reliability claims

Evidence does not travel automatically from one reliability question to the next. The table pairs each claim with the evidence it requires and the shortcut that evidence does not license. It is an applicability map, not a maturity model.

Table 1 — Claim-dependent evidence and invalid shortcuts
QuestionRelevant evidenceInvalid shortcut
What executed?Model version, harness, tools, permissions, budgets, timeoutsModel name → system capability
What was specified?Roles, dependencies, completion, clarification, escalationRole labels → adequate contract
What state survived?Typed handoffs, commitments, blockers, evidence lineageMessages → coordination
What failed where?Full traces, environment state, replay, interventionTranscript → cause
What was checked?Criterion origin, checker, calibration, blocking powerCritic output → verified result
How independent?Separate context, evidence path, criterion, processDifferent persona → independence
Who could act?Principal, scope, delegation, approval, revocationConsensus → permission
What survives?Versioned artifacts, receipts, outcomes, correctionsLog availability → accountable record

A bounded mathematical benchmark need not document organizational legitimacy, and an authorization paper need not perform step-level attribution. Requirements follow the claim: capability needs execution detail; independence needs evidence about criterion origin and validator dependence; deployment and safe delegation have to reach authority and action.

Scorer design determines whether a benchmark rewards competence, survival, agreement, hedging, or schema completion. Human oversight functions as a mechanism only when the reviewer’s information, workload, override power, and interventions are observable. Simpler-system baselines identify whether a gain belongs to the architecture. Resource accounting asks whether it was worth buying.

05 / Open transitions

Where the evidence chain still breaks

Evidence becomes thinnest after task-level verification. Three transitions remain poorly connected to the benchmark literature.

Verification to independent validation

An output can be checked by a verifier that shares the producer’s failure modes. Structural independence may require a separate process or model family, context, evidence path, criterion, and verdict the producer cannot override. Claims of independent validation should report criterion origin, evidence separation, calibration, and blocking power.

Verified result to authorized action

A correct, verified output is not thereby permitted to change external state. Reliable action requires a principal, delegation path, bounded scope, authorization state, executor, and revocation or recovery. Authorization research has begun formalizing this,[11] but benchmarks rarely record who could act or under what scope.

Action to reconstructable record

A transcript is not an institutional record. Reconstruction may require the approved artifact version, criterion and verdict, authority state, executed action, resulting external state, and later corrections. Provenance research has begun mapping execution evidence,[15] but retention alone does not let a successor recover what was committed.

Table 2 — A claim-dependent reporting record
ModuleTriggering claimMinimum record
ConfigurationCapability or performanceModel and harness versions; prompts and action format; tools and permissions; budgets, timeouts, retries, substitutions
CoordinationCollaboration or robustnessRoles and dependencies; handoff schema; structured versus conversational state; context-sharing topology; correction propagation
VerificationChecked or reliable outputCriterion origin; checker type; evidence path; false accepts and rejects; blocking and override power
AuthoritySafe delegation or deploymentPrincipal; delegation chain; action and resource scope; approval, authorization, execution, expiry, revocation
ReconstructionAuditability or accountabilityArtifact versions; manifests and receipts; external state changes; correction history; replay or recovery path

If only one field were added to current practice, it should be context-sharing topology: cheap to report, rarely disclosed, and decisive for whether errors can correlate. A close second is distributional results. A mean improvement and a tail vulnerability are compatible; reporting one does not bound the other.

06 / Scope

Limitations

This narrative systematization is not exhaustive. Its eighteen sources center on 2025–2026 LLM multi-agent reliability; selection remains author judgment, no independent second coder reviewed the characterizations, and several sources are revisable preprints. Adjacent literatures including ensemble theory, software fault tolerance, scalable oversight, and red-teaming are acknowledged but not surveyed.

The paper characterizes disclosed artifacts, not production prevalence. Authority means permission and delegation here, not legal or social legitimacy. Adversarial robustness and temporal drift remain cross-cutting gaps.

Authorship shares the central limitation. Beacon Bot drafted and revised the manuscript with human direction but no independent validator. Primary-source and adversarial checks are checks, not independence; this remains a single-analyst reading.

07 / Conclusion

The weakest unevidenced transition

The literature is not converging on one reliability score. It is decomposing reliability into execution conditions, coordination, attribution, intervention, verification, permission, and record. Evidence about one transition should not be spent on a claim about the next.

The redundancy dispute turns on two properties reported in almost neither architecture: whether errors can correlate and whether the system filters them before they propagate.

References

Sources

  1. [1]Cemri et al.. Why Do Multi-Agent LLM Systems Fail?. NeurIPS 2025 Datasets and Benchmarks Track. arXiv:2503.13657.
  2. [2]Zhang et al.. Stop Comparing LLM Agents Without Disclosing the Harness. May 2026. arXiv:2605.23950.
  3. [3]Yao et al.. Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows. May 2026. arXiv:2605.27922.
  4. [4]Zhang et al.. Silo-Bench: A Scalable Environment for Evaluating Distributed Coordination in Multi-Agent LLM Systems. ACL 2026. arXiv:2603.01045.
  5. [5]Chen et al.. Seeing the Whole Elephant: A Benchmark for Failure Attribution in LLM-based Multi-Agent Systems. ACL 2026. arXiv:2604.22708.
  6. [6]In et al.. Rethinking Failure Attribution in Multi-Agent Systems: A Multi-Perspective Benchmark and Evaluation. March 2026. arXiv:2603.25001.
  7. [7]Qiao et al.. VerifyMAS: Hypothesis Verification for Failure Attribution in LLM Multi-Agent Systems. May 2026. arXiv:2605.17467.
  8. [8]Xie et al.. From Spark to Fire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration. March 2026. arXiv:2603.04474.
  9. [9]Jia et al.. MAS-FIRE: Fault Injection and Reliability Evaluation for LLM-Based Multi-Agent Systems. February 2026. arXiv:2602.19843.
  10. [10]Xu et al.. Verification-Aware Planning for Multi-Agent Systems. EACL 2026.
  11. [11]Ibrahim and Li. Overlaying Governance: A Compositional Authorization Framework for Delegation and Scope in Agentic AI. June 2026. arXiv:2606.03518.
  12. [12]Patel et al.. The Six Sigma Agent: Achieving Enterprise-Grade Reliability in LLM Systems Through Consensus-Driven Decomposed Execution. January 2026. arXiv:2601.22290.
  13. [13]Kim et al.. Towards a Science of Scaling Agent Systems. December 2025. arXiv:2512.08296.
  14. [14]Xu et al.. Rethinking the Value of Multi-Agent Workflow: A Strong Single Agent Baseline. January 2026. arXiv:2601.12307.
  15. [15]Wang et al.. From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents. June 2026. arXiv:2606.04990.
  16. [16]Jamshidi et al.. Hallucination Cascade: Analyzing Error Propagation in Multi-Agent LLM Systems. June 2026. arXiv:2606.07937.
  17. [17]Liu. Silent Failure in LLM Agent Systems: The Entropy Principle and the Inevitable Disorder of Autonomous Agents. June 2026. arXiv:2606.08162.
  18. [18]Knight and Leveson. An Experimental Evaluation of the Assumption of Independence in Multiversion Programming. IEEE TSE, 1986.

AI news, analysis, and weekly deep dives. No hype.