Why AI Agents Change Their Minds and How Open Source Concordance Traces the Drift

CTGT has open-sourced Concordance, a testing framework designed to pinpoint why multi-step artificial intelligence pipelines reach conflicting conclusions when processing the exact same inputs.

The Non-Determinism Trap in Automated Workflows

Enterprises expanding their generative artificial intelligence programs face a persistent engineering hurdle: systems that make different choices when confronted with the same scenario twice. While individual language models are well known for probabilistic randomness, engineering teams often assume that setting temperature parameters to zero or using deterministic inference frameworks solves the issue. Recent findings suggest the fragility runs deeper. In complex agentic architectures, the surrounding application code often introduces subtle variations into prompt payloads, quietly derailing multi-step reasoning.

To help developers isolate these points of failure, applied interpretability startup CTGT has released Concordance under an Apache 2.0 open-source license. The tool functions as an evaluation and diagnostic protocol, allowing teams to trace how information travels across complex workflows and isolate the exact moment an automated process drifts off course. Rather than offering a proprietary hosted platform, the team has released the testing protocol, reference code, and baseline datasets freely to the public, aiming to establish clearer diagnostic benchmarks for mission-critical software.

How Small Pipeline Shifts Yield Conflicting Decisions

CTGT, founded by researcher Cyril Gorlla following work at UC San Diego and Stanford, operates as an applied interpretability lab focused on high-stakes enterprise domains such as insurance and financial compliance. The development of Concordance stemmed directly from a recurring failure inside the company's internal policy analysis tools.

During an internal evaluation, researchers tasked a Qwen3.8-27B model running on SGLang's deterministic inference engine with parsing a nine-page retail banking compliance manual. The pipeline processed the exact same document across four independent evaluation runs. The results should have been identical. Instead, the runs extracted 32, 31, 33, and 33 legal obligations respectively. In one instance, the software omitted an explicit directive mandating that staff escalate material infractions directly to the chief compliance officer.

Crucially, the underlying model was not behaving randomly. Across 74 identical queries executed throughout the run, the language model returned bit-for-bit identical responses. The breakdown occurred inside the surrounding application logic.

A granular investigation revealed three distinct pipeline vulnerabilities. First, the application had injected dynamic, upload-specific document identifiers directly into the prompt templates. Second, a 30-item internal routing list failed to maintain a fixed sort order, subtly changing the sequence of text presented to the model. Third, an internal topic registry accumulated state between calls, expanding unpredictably and altering the context window for subsequent prompts.

In one specific case, minor modifications across just four identifier lines within a 12,525-character payload altered the model's interpretation entirely. The model reclassified five distinct conflict-resolution protocols as mere background context, dropping an escalation rule that another run captured. Once developers pinned the identifiers, sorted the routing lists deterministically, and isolated each run within an unpolluted testing container, five consecutive evaluations returned an identical count of 34 obligations.

Isolating Prompt Drift From Model Variation

Debugging multi-agent systems often forces engineers to untangle two interwoven problems. A model might generate varying text because of server-side inference quirks, or the surrounding software orchestrating the model might inadvertently alter the payload.

Recent work from research teams such as Thinking Machines Lab has demonstrated that standard large language model serving setups rarely achieve absolute reproducibility out of the box. Small changes in batching schedules on modern graphics processing units alter floating-point math, leading to divergence even when sampling randomness is minimized. Systems such as SGLang combat this by introducing deterministic batching modes. In CTGT's validation benchmarks across 800 identical API calls, the underlying model achieved perfect textual consistency under deterministic runtime controls.

Yet perfect reproducibility does not equal accurate reasoning. When the compliance pipeline sent a corrupted prompt that misclassified a regulatory rule, replaying that exact request repeated the identical mistake eight times in a row. The software had achieved mechanical stability without achieving operational correctness.

Concordance approaches this challenge by decoupling system verification into structured checkpoints. It logs the explicit request payloads, intercepts raw model outputs, and compares intermediate artifacts stage by stage. By aligning traces side by side, developers can verify whether an erratic output is the result of non-deterministic inference or whether an upstream parsing script slightly shifted the input text.

The Emerging Role of Constrained Decision Models

As organizations seek higher reliability, software architecture is evolving beyond free-form text prompting. An emerging school of thought suggests that generative models should not be tasked with arbitrary decision paths or loose string matching. Instead, organizations are experimenting with specialized decision models and constrained decoding runtimes.

Frameworks such as TypeSafe's System One architecture, powered by models like Jev, along with decision-model frameworks highlighted by cloud providers like Amazon Web Services, deliberately restrict output token vocabularies. Instead of generating conversational summaries or open-ended JSON, these specialized models output strict numerical scores, bounded classifications, and probability distributions that standard deterministic code can evaluate.

Under such an architecture, a model might be tasked only with rating the semantic similarity between two policy clauses, while deterministic business logic explicitly handles the escalation path. If an infraction score crosses a set threshold, native software code executes the alert, removing the agent's ability to arbitrarily overlook a required step.

However, architectural constraints cannot entirely eliminate upstream context shifts. As technical documentation for constrained models frequently notes, classification layers remain sensitive to context ordering and prompt formatting. If an early document parser misidentifies a paragraph or drops an appendix before the text ever reaches a classifier, downstream schema restrictions cannot prevent a failure. The combination of constrained decision architectures and end-to-end trace auditing provides a more comprehensive defensive posture.

Navigating the AI Observability Landscape

The release of Concordance adds an open diagnostic methodology to a rapidly crowding observability market. Enterprise operations already use tools such as LangSmith, Braintrust, Langfuse, and Arize Phoenix to track latency, inspect tool usage, log execution graphs, and evaluate prompt performance.

Many of these commercial platforms already offer robust multi-run evaluation suites, session replay capabilities, and OpenTelemetry-compatible tracing pipelines. Rather than presenting itself as an all-in-one observability suite designed to replace existing infrastructure, Concordance functions primarily as a comparative debugging protocol. It offers a standardized method for running reproducible differential tests on complex data pipelines, identifying the precise divergence point between two runs.

As reported by VentureBeat, CTGT aims to position the Apache-licensed framework as a complementary component that can sit alongside existing telemetry stacks without requiring enterprises to rip out current tools.

For technical leaders, the immediate takeaway is practical: mission-critical automation cannot be verified solely by checking overall pipeline success rates. Every model interaction that creates an auditable record, signs off on an expense, or routes an enterprise ticket must be captured with complete payload fidelity. Maintaining static evaluation datasets with verified ground truths is essential to catching subtle logic drift whenever underlying prompt templates, embedding models, or orchestration libraries are updated.

Why it matters

For industrial deployments, field operations, and data centre management, agentic automation is quickly shifting from advisory assistance to direct operational control. Telecommunications operators and network teams are beginning to trial AI agents for automated optical path provisioning, fault diagnosis, and edge compute resource allocation. In industrial automation, autonomous systems parse safety documentation, interpret maintenance telemetry, and trigger dispatch tickets for field engineers.

When an AI system operates in an automated data centre or network operations centre, a non-deterministic pipeline error is not a minor cosmetic discrepancy. If an agent misclassifies an alert priority, alters maintenance thresholds because a prompt list reshuffled, or misses a critical safety escalation step, the operational consequences can cascade into service outages or physical equipment damage.

By enforcing strict input normalization and differential pipeline tracing, engineering teams can verify that automated dispatch systems, field diagnostic assistants, and infrastructure orchestration bots execute instructions with the exact same rigor as traditional compiled software. High-reliability operations cannot tolerate stochastic policy enforcement, making transparent pipeline auditing a mandatory foundational layer for the autonomous enterprise.