Data Quality, Metadata, Lineage, and Master Data

The last two lessons were about people and accountability: what governance is, and who is responsible. But accountability needs something to act on. A data owner who is "responsible for data quality" needs a concrete definition of quality, a way to measure it, and tools to manage it. This lesson covers that machinery, the working parts that turn governance intentions into operational reality.

Think of it as the difference between deciding "we will keep the building safe" and actually having smoke detectors, fire exits, inspections, and extinguishers. Governance roles set the intent; the machinery in this lesson is the equipment that carries it out.

There are four core pieces, each answering a practical question:

• Data quality: is the data good, and how do we know?
• Metadata and catalogs: what data do we have, and where is it?
• Data lineage: where did this data come from, and where does it go?
• Master data management: what is the single, authoritative version of key data?

And a fifth, more modern piece, data contracts, ties them together by pushing quality upstream to the source.

These are not separate products to buy but capabilities a governance program builds. Understanding them turns "data governance" from an abstract idea into a concrete set of things an organization actually does to make its data trustworthy, findable, and consistent. This is where governance meets the data itself.

"Data quality" sounds vague until you break it into specific, measurable dimensions. The field, following frameworks like DAMA's, defines quality along several standard axes, and naming them turns a fuzzy goal into concrete, checkable criteria.

The core dimensions:

• Accuracy: does the data correctly represent the real-world thing? (Is the recorded address the customer's actual address?)
• Completeness: is all the needed data present, with no missing values? (Do all customer records have an email?)
• Consistency: does the data agree with itself across systems? (Does the customer's status match in sales and support?)
• Timeliness: is the data up to date and available when needed? (Is the balance current or a week old?)
• Validity: does the data conform to the required format and rules? (Is the date a real date; is the email well-formed?)
• Uniqueness: is each real-world thing represented only once, with no duplicates? (Does this customer appear once, not five times?)

The power of these dimensions is that each can be measured as a rule and a score. "Email completeness" becomes: what percentage of customer records have a non-empty, valid email? That yields a number you can track over time, set a target for, and hold an owner accountable to. This is how the abstract responsibility "keep the data good" becomes concrete: define the dimensions that matter for a given dataset, write rules, and measure them.

The practice of scanning data to measure these dimensions is data profiling, and the ongoing checks are data quality rules. Together they convert quality from an opinion ("the data seems bad") into a metric ("address accuracy is 82 percent, below our 95 percent target"), which is exactly what an accountable owner needs to manage it.

You cannot govern data you cannot find, and in a large organization, simply knowing what data exists and where is a genuine challenge. The solution is metadata and the data catalog built from it.

Metadata is data about data: information describing a dataset rather than the content itself. It comes in a few types:

• Business metadata: what the data means in business terms, the definition of "active customer," who owns it, how it may be used.
• Technical metadata: the structure, where it is stored, its format, field types, how systems connect to it.
• Operational metadata: how the data is used and behaves, when it was last updated, how often it refreshes, who accesses it.

A data catalog collects this metadata into a searchable inventory, essentially a map and search engine for the organization's data. With a catalog, someone can search "revenue by region," find the authoritative dataset, see its definition, learn who owns it, check how fresh it is, and understand whether they are allowed to use it, all without hunting through systems or asking around.

The catalog is foundational because it enables everything else. It makes data discoverable (people can find and reuse trustworthy data instead of rebuilding it), it makes governance visible (definitions, owners, and rules live in one place), and it is the surface where quality scores, lineage, and access rules are surfaced to users.

The guiding principle is simple: you cannot manage, trust, or reuse what you cannot find. Metadata and the catalog solve the findability problem, turning an opaque sprawl of systems into a navigable, documented landscape. Without them, governance is blind, because no one has a complete picture of what data even exists to be governed.

Data rarely sits still. A number in an executive dashboard may have traveled through a dozen systems, joins, and transformations to get there. Data lineage is the record of that journey: where data came from, how it was transformed, and where it flows.

Think of lineage as a family tree or a supply chain for data. It traces a piece of data backward to its origins (which source systems, which calculations produced this figure) and forward to its destinations (which reports and models depend on it). A complete lineage lets you follow any number from the screen all the way back to the raw data it came from.

This capability is quietly essential for three reasons:

• Trust and debugging: when a number looks wrong, lineage lets you trace it back to find where the error entered, instead of guessing. Without lineage, diagnosing a bad figure across many systems is nearly impossible.
• Impact analysis: before changing a source system or a definition, lineage shows everything downstream that would be affected, so you do not silently break twenty reports by altering one field. You can see the blast radius before you act.
• Audit and compliance: regulators increasingly demand proof of where reported figures come from. Lineage provides that provenance, showing exactly how a regulatory number was derived.

Lineage connects directly to the roles from the last lesson: when a steward investigates a quality problem, lineage is the tool that shows them the path to the source. And it reinforces the 1-10-100 principle, because lineage helps you find and fix errors near their origin rather than after they have propagated everywhere.

The essential idea: lineage makes data's history transparent. In an ungoverned environment, numbers appear with no traceable origin and must be taken on faith; with lineage, every figure can be traced, explained, and verified, which is a precondition for genuinely trusting data at scale.

Return to the opening problem of the whole cursus: three departments, three customer counts. The deepest structural fix for that is master data management (MDM), the discipline of creating a single, authoritative version of the core data that the whole organization shares.

Start with a distinction between kinds of data:

• Master data: the core business entities everyone refers to, customers, products, employees, locations. These are the nouns the business is built on, used across many systems.
• Reference data: standard classification values, country codes, currency codes, status categories.
• Transactional data: the records of events, an order, a payment, a click, which reference the master data.

The problem MDM solves is that master data gets fragmented across systems. The same customer exists in the sales system, the support system, and the billing system, each with slightly different, separately maintained versions, different spellings, an old address in one, a typo in another. Now "how many customers" has no single answer because the customer exists many times over.

MDM creates a golden record: a single, reconciled, authoritative version of each master entity, built by matching and merging the scattered copies. The process involves matching (recognizing that "Bob Smith" in one system and "Robert Smith" in another are the same person) and deduplication (collapsing the duplicates into one trusted record) to establish one version everyone can rely on.

This is the direct antidote to the opening scene. With MDM, "customer" has one authoritative definition and one reconciled count, because there is a single source of truth rather than five competing copies. MDM is often the hardest and highest-value governance capability, precisely because master data is the shared foundation that the inconsistency problems of the whole cursus trace back to.

With the pieces defined, how does quality management actually run day to day? The mature approach follows one guiding principle from the first lesson: it is far cheaper to prevent bad data than to fix it later, the 1-10-100 rule in action.

The operational cycle has four repeating stages:

• Profile: scan the data to measure its current quality across the dimensions, accuracy, completeness, and the rest, establishing a baseline. You cannot improve what you have not measured.
• Define rules: encode the standards as automatic checks. "Email must be present and valid." "Every order must link to a real customer." These rules make quality expectations explicit and testable.
• Monitor: run the rules continuously so problems are caught as they arise, not discovered months later in a broken report. Quality is tracked as ongoing scores, not a one-time cleanup.
• Remediate at the source: when issues appear, fix not just the bad records but the cause, the form that accepted invalid input, the process that created duplicates, so the same errors stop recurring.

That last stage is where the 1-10-100 rule pays off. The instinctive response to bad data is to clean it downstream, scrub the report, fix the dashboard, but that treats the symptom while the source keeps generating errors. The mature move is to push quality upstream, catching and preventing errors as close to their origin as possible, ideally at the moment of data entry.

This reframes quality from a reactive cleanup task into a preventive, continuous discipline. Rather than an annual data-cleaning project that is undone within months, quality becomes a set of always-on rules and a habit of fixing root causes, so the data stays trustworthy over time instead of decaying between cleanups. Prevention beats correction, and the whole machinery is arranged to make prevention possible.

The most modern piece of machinery pulls the whole lesson together by formalizing the upstream principle: the data contract.

A data contract is an explicit, agreed specification between the producer of data and its consumers, defining what the data will look like and guarantee. Like a contract between a supplier and a customer, it states the terms in advance:

• the schema: the exact structure and fields the data will have,
• the quality guarantees: completeness, validity, and accuracy the consumers can rely on,
• the semantics: what each field means, so there is no ambiguity,
• the service level: how fresh and available the data will be.

The problem this solves is common and painful. In many organizations, a data-producing team changes something, renames a field, alters a format, stops populating a column, without warning, and silently breaks every downstream report and model that depended on it. The consumers only discover it when their dashboards break. A data contract prevents this by making the producer's commitments explicit and enforceable: changes that would violate the contract are caught automatically, before they break anything.

Data contracts are the natural companion to the domain-ownership and data-as-a-product ideas from the last lesson. If each domain treats its data as a product served to internal customers, the data contract is that product's specification and warranty. It shifts responsibility for quality firmly to the producer and makes it verifiable.

This completes the machinery. Quality dimensions define good data; catalogs make it findable; lineage traces its journey; master data gives a single source of truth; and data contracts push quality and reliability to the source and make them enforceable. Together these are the concrete tools through which governance accountability becomes trustworthy, usable data, the operational reality behind the policy. The final lesson turns to the other half of governance: keeping that data safe and properly controlled.

Quality dimensions define good data, catalogs make it findable, lineage traces its journey, master data reconciles it into one source of truth, and data contracts push quality to the source and make it enforceable.