Perspective
When Bad Data Sounds Confident
LLMs hide quality errors in fluent prose.
July 29, 2026 · Perspective · Aeolus Data Solutions
When we ask teams how they know their data is ready for AI, the answer is almost always “tests.” They point to hundreds of them: a dbt suite running on every merge, freshness checks on critical sources, and alert channels that never go silent.
For the systems those tests were built to protect, this is a great answer. The problem is that those tests were designed for a specific kind of consumer: one who complains.
In a traditional BI stack, a wrong number on a dashboard triggers a finance lead to notice the quarter looks impossible. A thread opens. A fix is deployed. That human feedback loop provides more real-world quality assurance than the test suite ever did. But that loop disappears the moment you switch to a model.
Garbage in, garbage out undersells the problem
The phrase “garbage in, garbage out” implies symmetry: bad input leads to visibly bad output. With language models, the reality is more dangerous because the “badness” is laundered into fluent prose.
We now have direct evidence of this. A March 2025 study presented at ACL 2025 used Wikipedia snapshot diffs (96,124 QA pairs across 219,463 documents) to test what happens when retrieval surfaces outdated information.
The results were stark: mainstream models saw performance drops of 20% or more even when the correct, current information was successfully retrieved, simply because outdated passages were present in the context. A single outdated passage dropped a 70B model’s overall score by more than 24%. Some configurations performed worse than random.
The most critical finding wasn’t the size of the drop, but the shape of the failure. When the model retrieved only outdated evidence, it answered confidently and incorrectly. When it retrieved nothing, it was appropriately uncertain.
Bad input doesn’t make a model hesitate; it makes the model sure. Your pipeline’s failure signal has shifted from a human saying “that number looks off” to a model writing a polished, incorrect paragraph.
The search for a canonical list of data quality
Most discussions on data quality rely on a numbered list of dimensions: accuracy, completeness, validity, timeliness, and so on. However, the confident tone of these lists masks a total lack of industry agreement.
| Source | Count | Dimensions |
|---|---|---|
| DAMA UK, 2013 | 6 | Completeness, uniqueness, timeliness, validity, accuracy, consistency |
| DAMA DMBOK, 2024 revision | 9 | DMBOK2 eight plus currency |
| ISO/IEC 25012:2008 | 15 | Split into inherent and system-dependent characteristics |
| ISO 8000-8:2015 | 3 | Syntactic, semantic, pragmatic (a measurement framework) |
Four reputable sources, four different answers. These are useful working vocabularies, not standards you can be measured against.
Aeolus view — Stop searching for a universal data quality checklist. In production AI, quality is defined by the specific failure modes of your retrieval chain, not a DAMA dimension.
A common example is the “seven pillars of data quality,” often attributed to the vendor Profisee. If you read the Profisee page actually cited, they list seven dimensions (uniqueness, completeness, consistency, precision, conformity, timeliness, and integrity), but they never use the phrase “seven pillars.” The list circulating under that name is neither a vendor standard nor an official authority. It is a reasonable checklist, but not a law.
Where the testing actually stops
Forget the vocabulary and look at the path a fact takes from a source system to a model’s answer. The real question isn’t which dimensions you cover, but how far along that path your coverage extends.
A fact's path from source system to model answer, and where conventional data tests reach
Coverage is stated in words on every stage rather than by colour, because the distinction matters more than it reads.
-
Source systems
Application databases, SaaS exports, event streams.
Conventional tests reach here
-
Ingestion and landing
Row counts, schema drift, load failures.
Conventional tests reach here
-
Transformation
Where not_null, unique and accepted_values live.
Conventional tests reach here
-
Warehouse tables and metrics
Freshness thresholds, contracts, reconciliation.
Conventional tests reach here
-
Existing test coverage stops here
Documents and unstructured sources
Contracts, tickets, wikis, PDFs. Often never modelled at all.
No conventional test here
-
Chunking and embedding
A clause split across two chunks is still a valid row nowhere.
No conventional test here
-
Retrieval
What actually got pulled into the context for this question.
No conventional test here
-
The answer
Fluent regardless of what any of the above returned.
No conventional test here
Everything above the line is a “table problem.” The industry has spent fifteen years solving this. Everything below the line is where AI features live, and most teams have zero automated checks here. This isn’t due to carelessness; it’s because the tools barely exist.
Applying classic checks to new stages
The fundamentals of data quality are still correct; they just need to be pointed at a different layer. Ben Rogojan makes a strong case for these fundamentals. Here is how they translate to the AI stack:
Range checks. In a warehouse, you alert if a “age” column suddenly hits 10,000. In retrieval, you should monitor the distribution: how many chunks are returned per query? How similar is the top result? How often does the best match fall below the threshold you set six months ago? These are numbers with expected ranges that almost nobody alerts on.
Category checks. In a table, you check if a state field contains non-state values. In a document set, you check if the collection has quietly acquired a class of content it was never meant to contain. Anomalo is a clear example of this approach, evaluating document collections for length, duplicates, topic, tone, and PII before they reach the model.
Freshness. This is the most critical check. dbt source freshness is excellent for tables, but it doesn’t know if the policy document you indexed in March was superseded in June. An index doesn’t go stale loudly; it goes stale silently and continues to provide confident answers.
This is compounded by a nuance found in the SIGIR 2024 paper “The Power of Noise”: documents that are semantically related but contain the wrong answer hurt accuracy more than plainly irrelevant ones. In some cases, adding random unrelated documents actually improved accuracy by up to 35%. The dangerous document isn’t the one that’s off-topic. It’s the one that looks exactly like the right answer but is a year out of date.
The reality of your tooling
The floor for data quality is lower than most teams assume.
dbt provides four built-in generic tests: unique, not_null, accepted_values, and relationships. Anything beyond that is custom-written. Saying “we have dbt tests” describes a habit, not a coverage level.
Model contracts are a powerful but underused tool. While dbt can fail a build if columns and types mismatch, constraints are handled differently based on the platform. For example, Spark does not enforce not_null or check at all. Databricks does, but applies them via ALTER after table creation, meaning bad data can still enter the table before the constraint fails.
The storage layer is even thinner. Apache Iceberg has no CHECK constraint mechanism and no primary/foreign key enforcement. It only supports required vs optional fields. Furthermore, compatible schema evolution does not include turning an optional column into a required one—that is an incompatible change. Open table formats provided portability, but not integrity.
Tooling specifically for retrieval quality remains immature. Most “RAG quality” tools are actually general LLM observability platforms. Arize Phoenix is a notable open-source example, but the commercial market is consolidating rather than maturing: Cisco acquired Galileo in May 2026, and Datadog acquired Metaplane in April 2025.
The cost of the detection gap
When bad input produces confident output, the time between the onset of a problem and its detection is where the most damage occurs.
Severity of business impact, by how quickly the data issue was detected
Share of respondents reporting significant or severe business impact.
| Item | Share reporting significant or severe impact |
|---|---|
A Monte Carlo/CDO Magazine survey found that 61% of respondents frequently experienced incidents where monitored metrics looked normal while a critical issue was underway. While these are vendor-run numbers, they align with the research: the gap between confidence and readiness is a systemic risk.
This disconnect is evident in the 2026 State of Data Integrity and AI Readiness study. 88% of leaders claimed their data readiness was “adequate” for AI, yet 43% simultaneously named data readiness as a top obstacle. Similarly, dbt Labs’ 2026 community survey showed that 71% of respondents worry that hallucinated outputs are reaching stakeholders.
Compliance is now a data quality problem
The regulatory landscape shifted significantly in July 2026.
When data governance obligations for AI actually bite in the EU
-
EU AI Act enters into force
-
AI Office publishes the GPAI training-content summary template
Article 53(1)(d) obliges providers to publish a summary using it.
-
General-purpose AI model obligations apply
-
The Digital Omnibus on AI enters into force
Regulation (EU) 2026/1744 defers the high-risk deadlines below.
-
Article 10 applies to Annex III high-risk systems
-
Article 10 applies to Annex I high-risk systems
Article 10 of the EU AI Act effectively codifies data quality for high-risk AI systems. It is often misquoted as requiring data to be “free of errors,” which would be an impossible standard. The actual text of Article 10(3) states:
Training, validation and testing data sets shall be relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose.
The phrase “to the best extent possible” shifts the obligation from perfection to diligence. However, Article 10(2) requires rigorous data governance practices covering design choices, origin, preparation, and bias mitigation. These must be documented under Article 11 and Annex IV. Evidence trails are difficult to produce retroactively.
Timing is critical. The Digital Omnibus on AI, Regulation (EU) 2026/1744, entered into force on 27 July 2026. It defers high-risk obligations to 2 December 2027 (Annex III) and 2 August 2028 (Annex I).
Note two caveats: Article 10 applies to high-risk systems, not general-purpose models (which have lighter transparency duties under Article 53(1)(d) via the AI Office’s template). Furthermore, a deferral is not a reprieve; building a documented governance trail takes quarters, making 2027 deadlines a 2026 project.
A note on the numbers
Two common AI failure statistics are frequently cited without context.
The claim that “87% of data science projects never reach production” comes from a 2019 VentureBeat write-up of a conference panel; there is no published methodology. Similarly, the “85% of AI projects fail” statistic is often cited without a primary source or a defined metric for failure.il” stat is a misreading of a 2018 Gartner forecast predicting that 85% of projects would deliver erroneous outcomes due to bias by 2022. “Erroneous” is not “failed.”
Gartner’s more recent data is more reliable: 63% of organizations lack or are unsure of their AI data management practices (July 2024 survey of 1,203 leaders). However, their accompanying claim about abandoning 60% of AI projects is an analyst prediction, not a measured observation.
Aeolus view — Pointing to a test suite to prove AI readiness is a natural instinct, but those tests were built for a user who pushes back. You are now handing your data to a model that never will. We recommend starting in an unglamorous place: pick the three most critical questions your AI feature will answer, trace the exact documents and tables that provide those answers, and ask if your stack would alert you if one of those sources went stale. Usually, it wouldn’t. Closing that gap is cheaper than another round of model selection, and it is the only way to ensure the feature can be trusted.
Where this leaves you
The reality is simple: your existing tests are good, but they stop at the warehouse. AI features consume data in the stages beyond—documents, chunking, and retrieval—where tooling is still immature.
Because failure in these stages is silent and confident, detection lag (not error rate) is your primary risk. Moreover, for high-risk systems in the EU, the record of how you manage this becomes a legal artifact by late 2027, making it a 2026 priority.
You don’t need a new platform. You need to extend the checks you already know how to write into the parts of the stack you haven’t been asked to cover yet.
We build and advise on this specific layer. If you need a second opinion on your data’s actual readiness before an AI feature depends on it, we are happy to help. If your data isn’t ready for this yet, we’ll help you get there. We have written extensively on why enterprise AI runs on data engineering, treating metrics as build artifacts, and the new layer of the 2026 stack.
Want a second opinion on your data stack?
Every Aeolus engagement starts with a fixed-fee data & AI-readiness audit — a short, low-risk first step before any larger build.
Book a data & AI-readiness audit