--- title: 'Canonical Data Mesh: scaling data governance' description: Scale data governance with Canonical Data Mesh. Build reliable, observable, open-source data platforms from ingestion to consumption url: https://canonical.com/blog/canonical-data-mesh-scaling-data-governance?format=md --- 1. [Blog](https://canonical.com/blog) 2. Article --- [Ali Kelkawi](https://canonical.com/blog/author/ali-kelkawi "More about Ali Kelkawi") 31 August 2026 # Canonical Data Mesh: scaling data governance [data-engineering](https://canonical.com/blog/tag/data-engineering) [observability](https://canonical.com/blog/tag/observability) --- Share the article Data governance is easy to describe and much harder to operate. Most organizations can define ownership, document policies, and agree that data should be easier to find and trust. The difficult part starts when those ideas have to work across real systems. A pipeline may be running successfully, but nobody knows who owns the source. A dashboard may answer an important business question, but its lineage is unclear. A query engine may provide access to many systems, but policy enforcement is inconsistent. Each individual tool may be working, while the overall platform remains difficult to understand, operate, and govern. At Canonical, this is the problem we are working to solve with the [Canonical Data Mesh](https://canonical-data-mesh-documentation.readthedocs-hosted.com/). Our approach starts from a simple principle: data governance should be built into the way data is produced, transformed, accessed and operated. We believe governed data should carry the context needed to understand and trust it throughout its lifecycle: clear ownership, lineage, quality, access policy and operational visibility. Governance is therefore not the responsibility of a single tool or team, but an outcome of how the whole platform works together. Our goal is not just to assemble a set of data tools. It is to make those tools work together in a way that is repeatable, observable, and practical to operate from initial deployment through day 2 operations. Built from open source tooling, the Canonical Data Mesh is structured around a lifecycle that starts with ingestion, carries context through orchestration and quality, publishes discovery and lineage, enforces access policy, and serves trusted data to end users. The following sections will detail each of these layers and the open source tools that power them, explaining how context and governance flow through the entire lifecycle. This architecture is guided by the core principles of data mesh: treating data as a product, placing ownership close to the business domains that understand the data, providing shared self-service platform capabilities, and applying governance through a federated model. The platform provides the common infrastructure and controls needed to support these principles without requiring every domain to build and operate its own data stack. *A visual representation of the Canonical Data Mesh architecture* ## Start with ingestion: bringing data into the mesh The lifecycle begins when data enters the platform. Data comes from SaaS systems, operational databases, files, APIs, and other business systems. In the Canonical Data Mesh, [Airbyte](https://airbyte.com/) provides the ingestion layer for this part of the platform. This is important because ingestion can easily become a collection of one-off scripts, custom API jobs, and manually maintained pipelines. That approach may work at a small scale, but it becomes difficult to operate as the number of data sources grows. Each new system introduces another set of credentials, schedules, schemas, failure modes, and destination requirements. Airbyte gives us a common pattern for connecting to these systems. Sources can be configured through reusable connectors, sync schedules can be managed consistently, and data can land into destinations in a predictable structure. Instead of every team solving ingestion differently, the platform provides a shared way to bring data in. By standardizing how data is ingested, scheduled, and landed, Airbyte gives teams a cleaner starting point before data moves into transformation, quality checks, discovery, and reporting. ## Transform data into governed assets After ingestion, raw data needs to be cleaned, modeled, and shaped into something meaningful. [dbt](https://www.getdbt.com) is an open-source data transformation framework that allows teams to use SQL and software engineering practices to build, test, document, and maintain data models. In the Canonical Data Mesh, dbt is where raw sources start to become useful datasets, models, metrics, and reports. [Temporal](https://temporal.io) is an open-source workflow orchestration platform used to coordinate and reliably execute data workflows. We use it to schedule and run transformation jobs more reliably, with durable execution for long-running or failure-prone processes. At this point, raw data starts to become something the organization can depend on. A governed asset is not just a table or a dashboard. It is an asset with enough surrounding information to be trusted and operated over time: ownership, lineage, quality expectations, access policy, and observability. Observability matters because data assets continue to change after they are published. Pipelines fail, schemas evolve, freshness drifts, usage patterns shift, and downstream dependencies grow. Making those signals visible helps owners understand when an asset needs attention, helps consumers judge whether it can be trusted, and helps platform teams understand the impact of incidents or changes. Transformation is also one of the richest sources of lineage. When dbt models are connected into the metadata layer, users can see not only where data landed, but how it changed on the way to consumption. That matters when people are trying to understand a metric, debug a broken report, or assess the impact of a schema change. ## Validate trust with Soda Data quality issues are often simple, but they can have a large downstream impact. A table might be late, a key column might become unexpectedly null, row counts might drop after a transformation, or a model might produce an empty result while the dashboard still loads. [Soda](https://soda.io/) provides the data quality layer in the Canonical Data Mesh. It gives teams a way to define checks that answer practical questions, such as: is the data fresh? Are required fields complete? Did the number of rows change unexpectedly? Are values within the expected range? These checks are most useful when they are connected to the rest of the Canonical Data Mesh. A failed quality check should not only live inside a job log, but rather be visible to the data owners and consumers, helping with incident response and downstream impact analysis. In practice, quality checks are one of the clearest ways to make governance visible. Governance cannot establish trust in an asset if the platform cannot also show whether that asset is complete, current, and behaving as expected. ## Publish metadata and lineage in DataHub As data moves through ingestion, orchestration, transformation, and quality checks, the Canonical Data Mesh produces metadata that needs somewhere to accumulate. [DataHub](https://datahub.com) is an open-source metadata platform and data catalog that brings this information together. It helps teams discover data assets, understand ownership and documentation, and trace lineage across systems. For users, DataHub helps answer questions like: what dataset should I use? Who owns it? Where did it come from? What dashboards depend on it? For governance teams, it provides a place to reason about ownership, documentation, and dependencies. For platform teams, it reduces the manual effort needed to answer questions about where data came from and what might break if it changes. Metadata from different tools can help build a richer map of the platform. The goal is to reduce the number of questions that require manual investigation. A user should be able to find a dataset, understand how it was created, review its quality status, identify its owner, and see which reports depend on it without needing to ask around first. This reflects a core principle of our approach: governance context should accumulate as data moves through the platform, rather than having to be reconstructed manually afterwards. Ownership, lineage, documentation and quality signals become part of the asset’s operational context. ## Govern access at the point of query with Ranger and Trino Governance must also be enforceable. Discovering who owns a dataset or documenting who should have access is not enough; those policies need to be applied consistently at the point where data is used. In a distributed data platform, users need a consistent way to query data across different systems without moving everything into a single warehouse. At the same time, that access needs to be governed consistently at the point where data is used. [Trino](https://trino.io/) provides the federated query layer for the Canonical Data Mesh, allowing users to query distributed data sources where they live. [Apache Ranger](https://ranger.apache.org/) provides the policy enforcement layer, defining who can access what and applying those policies through Trino when users query data. Together, Trino and Ranger make federated access practical to govern. Trino gives users a common way to query data across different systems, while Ranger applies the access rules that determine what each user or group is allowed to see. This means data owners and governance teams can define policies without needing every underlying system to solve access control differently. Platform teams can then apply those policies through shared infrastructure, and users get a more consistent experience when working across data assets. The goal is not simply to make more data reachable, but to make distributed data access easier to manage, audit and trust. ## Consume trusted data in Superset [Apache Superset](https://superset.apache.org) provides the BI and dashboard layer for this part of the platform. Dashboards are often where trust is tested most visibly. A user may not care which pipeline loaded the data, which model transformed it, or which policy controls access to it. They care whether the number on the dashboard is correct, current, and safe to use for a decision. This is where the earlier parts of the platform become visible to the end user. Superset integrates with Trino, so dashboards can use the same federated query engine and access controls described above. Superset also integrates with DataHub, which can capture metadata about datasets, charts, and dashboards. That gives users a clearer path from a dashboard back to the data sources, models, and dependencies behind it. This is especially important for business-critical reporting. If a dashboard is used to make decisions, users should be able to understand what it depends on and whether it can be trusted. ## Operate the platform lifecycle with Juju and Charmed Operators Each component in the Canonical Data Mesh stack has its own operational needs. Services need configuration, authentication, secrets, storage, scaling, upgrades, observability, and integration with the rest of the platform. Without a consistent operating model, the platform becomes a collection of custom deployments and one-off scripts. This makes governance harder to sustain. We therefore treat operability as part of governance. Policies, quality controls and metadata integrations only remain effective if the underlying services can be deployed, upgraded, observed and integrated consistently over time. [Juju](http://juju.is) gives us a way to model applications and their relationships. [Charmed Operators](https://canonical.com/juju/charms-architecture) encode operational knowledge for each service. Together, they help us deploy, integrate, configure, scale, upgrade, and observe the platform from day 0 to day 2 in a consistent and repeatable way. That consistency matters in production. When services are operated through a shared model, platform teams have a clearer path for routine changes, upgrades, and support activities. The following example is deliberately simplified to focus on the operating model. A production deployment would also require configuration for identity, secrets, TLS, ingress, backup, storage, scaling, and observability, which are outside the scope of this example. Here, the point is simply to show how applications are deployed as charms and how integrations between them are expressed as Juju relations. ``` # Example: make a governed analytics path available to users # Ingest data juju deploy airbyte-k8s airbyte juju deploy postgresql-k8s metadata-db --channel 14/stable --trust juju integrate airbyte metadata-db # Query across governed sources juju deploy trino-k8s trino juju deploy hive-metastore-k8s hive-metastore juju integrate trino hive-metastore # Enforce access policy juju deploy ranger-k8s ranger juju integrate trino ranger # Publish discovery and lineage juju deploy datahub-k8s datahub juju integrate datahub trino # Enable data visualization juju deploy superset-k8s superset juju integrate superset trino # Check the full relation graph juju status --relations ``` The important point is not the exact sequence of commands. The value is the operating model. Each component is deployed with its own operational knowledge, and each integration is represented as a relation. This gives platform teams a repeatable way to connect ingestion, query, policy, metadata, quality, and consumption layers without rebuilding the same glue each time. More information on Charmed Operators can be found at [charmhub.io](http://charmhub.io). ## Secure access with the Canonical Identity Platform A governed data platform also needs a consistent approach to identity. Users may interact with the Data Mesh through query engines, dashboards, catalogs, APIs and future AI-enabled interfaces. Each of these entry points needs authentication, session management and a way to connect users and groups to downstream access decisions. The [Canonical Identity Platform](https://canonical-identity.readthedocs-hosted.com/) provides a modular identity and access management layer for this kind of environment. It can act as an identity provider in itself, or broker external identity providers such as Microsoft Entra ID, Okta, Google and GitHub to downstream services, creating a unified sign-on experience across workloads. In the Canonical Data Mesh, this helps provide a consistent identity foundation across tools such as Trino, Superset, DataHub, and supporting services. Identity becomes part of the platform operating model rather than something each component solves separately. That consistency becomes even more important as users interact with governed data through more entry points, including APIs, MCP servers, and AI-enabled analytics experiences. ## Observe and improve the platform Day 2 operations matter as much as the initial deployment. Governance degrades quickly when pipelines, catalogs, policies, and dashboards are operated as separate islands. Platform teams need visibility into both the services and the data paths between them: whether ingestion jobs are running, whether query engines are overloaded, whether dashboards are failing, whether catalogs are reachable, and whether users are experiencing problems. The [Canonical Observability Stack](https://documentation.ubuntu.com/observability/track-2/) (COS) provides the observability layer for metrics, logs, dashboards, and alerts. The Canonical Data Mesh Charmed Operators can expose operational signals through COS, giving platform teams a shared view of the stack. ## What we are learning One lesson from building this platform is that governance has to show up in the systems people actually use. Defining ownership in one system, lineage in another, and access policies somewhere else is not enough if those signals remain disconnected. Governance becomes more useful when that context appears where people query, build, monitor, and consume data. That principle has shaped the architecture of the Canonical Data Mesh. The platform connects the full path from Airbyte ingestion through to Superset and Trino consumption, with DataHub, Ranger, Soda, Temporal, dbt, and Hive metastore contributing the context and controls needed along the way. Making those components work together consistently is also an operational challenge. Juju and Charmed Operators provide the foundation for deploying, integrating, and operating this lifecycle in a repeatable way, helping governance remain sustainable as the platform evolves. ## Conclusion Scaling data governance depends on more than selecting the right tools. Those tools need to work together across the path data takes through the platform. As described throughout this article, the Canonical Data Mesh assembles open source tooling for structured ingestion, transformation, discovery, quality, access control, observability, and consumption. Juju and Charmed Operators help make the platform deployable and operable from day 0 to day 2. This foundation also opens the door to the next stage of data platform experience: making governed data easier to consume in the era of generative AI. That could include conversational analytics, AI-assisted discovery, automated impact analysis, and MCP-based integrations that let users and agents interact safely with Data Mesh tooling. If you want to explore the platform in more detail, the [Canonical Data Mesh documentation](https://canonical-data-mesh-documentation.readthedocs-hosted.com/) covers the architecture, components and deployment approach. We are continuing to evolve this approach as we scale governed data access across Canonical. Follow the [Canonical blog](http://canonical.com/blog) for future updates on our data mesh journey, Charmed Operators, open source data governance, and AI-enabled data platforms. ## Further reading * [Canonical Data Mesh documentation](https://canonical-data-mesh-documentation.readthedocs-hosted.com/) [Get in touch Interested in running Ubuntu in your organization?](https://ubuntu.com/about/contact-us/form) ## Sign up for our newsletter Get the latest Canonical news and updates in your inbox. Work email: \*I agree to receive information about Canonical's products and services. By submitting this form, I confirm that I have read and agree to [Canonical's Privacy Policy](https://canonical.com/legal/dataprivacy). Sign up ## Share on --- ## Related posts [### Building a dry-run mode for the OpenTelemetry Collector](https://canonical.com/blog/building-a-dry-run-mode-for-the-opentelemetry-collector) Teams continuously deploy programmable telemetry pipelines to production, without having access to a dry-run mode. At the same time, most organizations lack staging... [Simon Aronsson](https://canonical.com/blog/author/0x12b) 17 March 2026 [### Ubuntu AI podcast: Understanding MLOps and Observability](https://canonical.com/blog/ubuntu-ai-podcast-understanding-mlops-and-observability) Ubuntu AI podcast Welcome to Ubuntu AI podcast! From fun experiments to enterprise projects, AI became the center of attention when it comes to innovation, digital... [Hasmik Zmoyan](https://canonical.com/blog/author/hzmoyan) 29 September 2023 [### ML Observability: what, why, how](https://canonical.com/blog/ml-observability) Note: This post is co-authored by Simon Aronsson, Senior Engineering Manager for Canonical Observability Stack. AI/ML is moving beyond the experimentation phase. This involves... [Andreea Munteanu](https://canonical.com/blog/author/munteanuandreea) 6 July 2023 [### Observability vs. monitoring debate: An irreverent view](https://canonical.com/blog/observability-vs-monitoring-debate-an-irreverent-view) This post provides an irreverent view on the debate around observability vs. monitoring, which is a rather confounding topic. [Michele Mancioppi](https://canonical.com/blog/author/michele-mancioppi) 23 November 2021