Data Governance, Lineage and Data Mesh Explained

Data governance is the set of rules, roles and tools that make sure an organisation’s data is known, trusted, protected and used properly. In practice it answers everyday questions: Who owns this table? What does this column mean? Where do these numbers come from? Who may see customer emails? How long do we keep order history?

Data mesh is one way to organise who does that work. Instead of one central data team owning every pipeline and table, each business domain owns its data and publishes it as a product, on a shared platform, under shared rules.

What data governance covers

Area	The question it answers	Typical practice
Ownership	Who is responsible for this data?	Every important dataset has a named owner and a steward
Catalog and definitions	What exists, and what does it mean?	A searchable catalog with descriptions, tags and a business glossary
Lineage	Where did it come from, and what uses it?	Automatic capture of table-to-table and column-to-column dependencies
Quality	Can we trust it?	Tests, contracts and quality scores; see data quality
Access and privacy	Who may see what?	Roles, column masking, row policies; see protecting PII
Retention and deletion	How long do we keep it?	Retention rules per dataset, deletion processes
Compliance and audit	Can we prove all of this?	Access logs, approvals, policies in code

Roles in plain words:

• A data owner (usually a business leader) decides what the data is for and who may use it.
• A data steward looks after definitions and quality day to day.
• Data engineers build the controls: tests, lineage capture, access policies, retention jobs.

Catalogs and lineage

A data catalog is a search engine for your data: tables, columns, owners, descriptions, tags such as “contains PII”, popularity and freshness. Examples include Unity Catalog in Databricks, AWS Glue Data Catalog with Lake Formation, Microsoft Purview, Google Dataplex, DataHub and OpenMetadata.

Lineage records what each dataset is built from. It is captured by parsing SQL, by reading the dependency graph of tools such as dbt and Airflow, or by jobs emitting events in a standard format such as OpenLineage. Its two main uses are:

• Impact analysis (downstream): “If I change this source table, what breaks?”
• Root-cause analysis (upstream): “This report is wrong; where could the problem come from?”

Here is a small lineage graph and both questions:

from collections import deque

# Lineage: each table or report and the inputs it is built from (as a catalog would record it).
inputs = {
    "raw.crm_customers":       [],
    "raw.shop_orders":         [],
    "raw.payments":            [],
    "core.customers":          ["raw.crm_customers"],
    "core.orders":             ["raw.shop_orders", "raw.payments"],
    "mart.revenue_daily":      ["core.orders"],
    "mart.customer_ltv":       ["core.orders", "core.customers"],
    "report.finance_board":    ["mart.revenue_daily"],
    "report.marketing_ltv":    ["mart.customer_ltv"],
}
owners = {"raw.payments": "payments team", "core.orders": "data platform", "mart.customer_ltv": "marketing analytics",
          "report.finance_board": "finance", "report.marketing_ltv": "marketing"}

def downstream(node):
    """Everything built from this node: who is affected if it breaks or changes."""
    users = {n: [m for m, ins in inputs.items() if n in ins] for n in inputs}
    seen, queue = [], deque([node])
    while queue:
        for child in users[queue.popleft()]:
            if child not in seen:
                seen.append(child); queue.append(child)
    return seen

def upstream(node):
    """Everything this node depends on: where its numbers come from."""
    seen, queue = [], deque([node])
    while queue:
        for parent in inputs[queue.popleft()]:
            if parent not in seen:
                seen.append(parent); queue.append(parent)
    return seen

print("if raw.payments changes, affected:")
for n in downstream("raw.payments"):
    print(f"  {n:22} owner: {owners.get(n, 'unassigned')}")
print("report.marketing_ltv depends on:", ", ".join(sorted(upstream("report.marketing_ltv"))))

if raw.payments changes, affected:
  core.orders            owner: data platform
  mart.revenue_daily     owner: unassigned
  mart.customer_ltv      owner: marketing analytics
  report.finance_board   owner: finance
  report.marketing_ltv   owner: marketing
report.marketing_ltv depends on: core.customers, core.orders, mart.customer_ltv, raw.crm_customers, raw.payments, raw.shop_orders

A change to the payments feed reaches five datasets and four teams, which tells you whom to warn before the change ships. It also exposes a governance gap: mart.revenue_daily feeds the finance board but has no owner. Real catalogs do the same traversal over thousands of nodes.

Data mesh

Data mesh was proposed by Zhamak Dehghani in 2019 as an answer to a common problem: a central data team becomes a bottleneck, far from the business, maintaining pipelines for data it does not understand. It rests on four principles:

1. Domain ownership. The teams that create the data (orders, payments, marketing) own its analytical versions too.
2. Data as a product. Each published dataset is treated like a product: documented, tested, versioned, with an owner, a quality promise and users it serves.
3. Self-serve data platform. A central platform team provides the tools (storage, pipelines, catalog, access control) so domains can publish without building infrastructure.
4. Federated computational governance. Rules (naming, security, privacy, interoperability) are agreed centrally but enforced automatically by the platform, not by manual review.

When data mesh works

• Large organisations with many domains and a central team that has become a bottleneck.
• Domains with engineering capacity and a real wish to own their data.
• A strong platform team and leadership support for changing who owns what.

When it fails

• Small companies. With one data team of five, a mesh adds process without solving a problem.
• Domains without data skills, which are suddenly “owners” of pipelines they cannot maintain.
• No shared platform or standards, which produces silos with inconsistent definitions: the opposite of the goal.
• Renaming instead of changing. Calling existing tables “data products” without owners, contracts or quality promises.

Many organisations adopt parts of it, such as domain ownership of key datasets and data contracts, without a full mesh.

Common mistakes

1. Treating governance as a document instead of controls built into the platform: tests, policies, lineage, retention jobs.
2. A catalog nobody updates. Capture metadata automatically from the tools you already run.
3. No owners. Every important dataset needs a person to ask and a person to fix it.
4. Starting a data mesh as a reorganisation before there is a platform for domains to use.

What interviewers look for

For “what is data governance?” or “what is data mesh, and when does it fail?”:

• The areas of governance and the roles, in plain words.
• What lineage is, how it is captured and what you use it for.
• The four data mesh principles, and an honest view of when it does not fit.
• An example of a control you built: a test, a masking policy, a lineage integration.

Practise with the data engineering interview questions.

Key takeaway

Data governance makes data known, trusted and protected through ownership, catalogs, lineage, quality checks, access rules and retention. Lineage answers “what breaks if this changes?” and “where did this number come from?”. Data mesh moves ownership to business domains that publish data products on a shared platform under automated rules; it helps large organisations with a central bottleneck and hurts small ones. Next: pipeline observability.