The Illusion of Data Quality: When Every System Is Green but Reporting Is Wrong

Most enterprise data quality programs are built around systems.

The CRM enforces required fields and acceptable values. The ERP validates transaction structures. Data engineering monitors pipelines for failures, delays, and schema changes. When something breaks in one of those systems, alerts fire. From the perspective of each individual platform, there’s evidence of control.

But organizations still experience reporting that doesn’t add up.

The tension usually surfaces in cross-functional reporting rather than in technical incidents. A revenue review exposes a discrepancy between booked and recognized revenue. A customer metric differs depending on whether it’s pulled from an operational dashboard or a finance report. A KPI looks right in one context and wrong in another. In each case, the underlying systems are behaving according to their own rules. The issue isn’t corrupted data—it’s misalignment.

That’s the illusion of data quality.

Data can be valid within a system and still fail to represent a coherent version of the business once it’s integrated with other systems. Most enterprises have invested in vertical data quality — validation inside individual domains — but enterprise decisions depend on alignment across domains.

When “Correct” Data Produces Conflicting Outcomes

Enterprise reporting is rarely sourced from a single system. Revenue, for example, reflects inputs from CRM opportunity stages, billing platforms, contract terms, fulfillment data, and general ledger entries. Each system captures a legitimate slice of reality and applies its own definitions, timing assumptions, and inclusion logic.

When those datasets are brought together in a warehouse or lakehouse, they’re treated as though they describe the same concept. In practice, Sales may define revenue based on signed contracts, Finance based on recognized revenue, and Operations based on fulfillment status. None of these interpretations are incorrect — they were just never reconciled at the integration layer.

The result is not necessarily poor quality data. It’s multiple, simultaneous interpretations of the same metric. It’s data that’s not fit for purpose. Without explicit cross-system validation, those interpretations coexist until they collide in a PowerBI dashboard. 

Over time, teams compensate by maintaining side reconciliations, adding caveats to metrics, and relying on experience to interpret results. The systems remain technically healthy, but confidence in the integrated view, not to mention efficiency degrade.

What Vertical Data Quality Actually Solves

Don’t misunderstand. Vertical data quality is essential. Within a system, validation rules ensure that required attributes are present, identifiers follow expected formats, numeric values fall within acceptable ranges, and relationships between tables remain intact. These controls protect operational stability and prevent obvious corruption.

They are also relatively straightforward to govern because ownership and domain boundaries are clear.

However, vertical validation ends at the system boundary. It doesn’t test whether customer identifiers resolve consistently across platforms, whether revenue logic aligns between billing and finance, or whether status codes in one workflow contradict those in another. Those relationships exist between systems, and they require a different layer of data quality: horizontal data quality. 

How Horizontal Data Quality Extends Vertical Controls: Four Common Scenarios

Understanding what vertical data quality solves makes it easier to see where it falls short.

1. Entity Resolution Failures

Duplicate customers aren’t usually very dramatic inside a single system. Marketing may have “Jon Smith” tied to one email, while Support has “Jonathan Smith” with a slightly different record. Both entries pass validation because both systems are internally consistent. No alert here.

The issue only surfaces when those systems are treated as representing a single customer relationship. Without horizontal matching, the organization interacts with one person as if they were two (eg a review request goes out while an escalation is open).

Qualytics addresses this through Entity Resolution checks that detect fuzzy matches and homophones, not just exact IDs, along with cross-datastore Exists In / Not Exists In checks that validate whether entities truly align across systems rather than assuming they do.

2. Data Reconciliation Gaps Between Systems

Revenue discrepancies often stem from differences in definition rather than errors. A CRM may show $100K in closed deals while Finance shows $90K invoiced. Both numbers are defensible within their systems. The difference may reflect timing, exclusions, or accounting treatment. Without cross-system reconciliation, that gap becomes an unexplained variance rather than a governed condition.

Qualytics’ Data Diff reconciles datasets across systems at the record level, producing a clear view of raw differences. Metric checks compare aggregates across platforms and alert when values diverge beyond defined thresholds. For one healthcare customer, a controlled metrics table feeding regulatory reports and AI models is continuously validated this way, ensuring changes to aggregated views are detected within minutes rather than during audits.

3. Cross-System Business Logic Failures

A customer can appear “Active” in an application while being placed on “Credit Hold” in Finance. Each system is behaving correctly within its own workflow. No validation fails. Yet together, they create revenue leakage or compliance exposure.

MAPFRE’s experience illustrates the distinction between vertical and horizontal controls. Their quoting system generated over 18,000 validation rules that caught anomalies at the record level — strong vertical data quality. But the exposure risk they faced lived between billing, policy history, invoicing, and charge records. Only by validating logic across those systems could the pattern be detected.

Qualytics enables cross-datastore Satisfies Expression checks that formalize these relationships — for example, enforcing that if Finance Status = “Credit Hold,” then App Status cannot equal “Active.” Business logic that spans domains becomes an explicit control.

4. Structural Inconsistencies

Not all horizontal failures are about business definitions. Some are structural. One system stores dates as MM/DD/YYYY, another as DD/MM/YYYY. A value like “01/05/2023” passes validation in both systems but represents different months once aggregated. The pipeline runs successfully; the distortion appears only in reporting.

Qualytics’ Conformity checks learn expected patterns from historical data and flag structural shifts even when ingestion does not fail. A silent vendor-side format change can be detected before it cascades into regional revenue summaries or regulatory reports.

From Local Validity to Enterprise Trust

The illusion of data quality persists because it is easier to measure what happens inside a system than what happens between systems. Rule coverage, anomaly counts, and pipeline uptime are visible indicators. Cross-system alignment requires coordination across domains and shared ownership of definitions.

Moving beyond the illusion does not mean abandoning vertical data quality. It means extending it.

Vertical controls ensure that individual systems function reliably within their domains. Horizontal controls ensure that those domains describe a coherent business when viewed together. When integration layers are treated as control surfaces rather than passive consolidation points, discrepancies become detectable conditions instead of surprises.

As organizations centralize data and increase automation — particularly as AI systems consume integrated datasets — the cost of misalignment grows. Decisions are made faster and at greater scale. Small inconsistencies propagate more widely. The integration layer becomes more consequential.

Trusted data does not emerge from isolated green checks. It emerges from validated relationships across systems.

That’s the difference between data that is locally correct and data that represents a coherent enterprise reality.

‍