Components¶

Charmed Apache Kafka is more than a single charm: it is a family of operators, workload artefacts, and Terraform modules that together provide a fully managed Apache Kafka platform on both machines and Kubernetes. This page gives an overview of every component in that family, so you can tell at a glance what exists, what it does, and where to find it.

All charms are available individually on Charmhub and can also be deployed together with minimal configuration overhead via the Charmed Apache Kafka Terraform bundle.

Core components¶

The core of the platform is a single charm, kafka (and its Kubernetes counterpart kafka-k8s), that can take on three different roles through its roles configuration option:

broker: standard Apache Kafka broker functionality
controller: KRaft (Kafka Raft) metadata quorum node
balancer: Cruise Control node for partition rebalancing (co-located with broker or controller, not deployed on its own)

This means the KRaft controller and the Cruise Control balancer are not separate charms — they are deployments of the same charm with a different roles value. For example, a split deployment runs brokers and controllers as separate applications of kafka, connected through the peer-cluster-orchestrator integration. For more detail, see the roles option in the configurations reference and the unit management guide.

Component	`roles` value	Purpose
Apache Kafka broker	`broker`	Runs the message brokers that store and serve data
KRaft controller	`controller`	Manages cluster metadata through the Kafka Raft quorum
Cruise Control balancer	`balancer` (co-located with `broker` or `controller`)	Monitors the cluster and rebalances partitions

All three components run the same workload: the charmed-kafka snap on machines, or the OCI image on Kubernetes.

The source code for all four Kafka charms (machine and K8s, broker and Connect) lives in a single repository, canonical/kafka-operator. The former separate repositories (kafka-k8s-operator, kafka-connect-operator, and kafka-connect-k8s-operator) have been archived and are now read-only.

Cruise Control¶

Cruise Control is LinkedIn’s open-source system for streamlining the operation of large Kafka clusters. It continuously monitors cluster health and computes optimisation proposals for partition and replica placement, which can then be applied to rebalance the cluster.

In Charmed Apache Kafka, Cruise Control is bundled inside the charmed-kafka snap (as charmed-kafka.cruise-control) and enabled by adding the balancer role to an application that already runs the broker or controller role — the balancer is a co-located role and is not deployed as a separate application. For a step-by-step introduction, see the partition rebalancing tutorial and the partition reassignment guide.

Kafka Connect¶

Kafka Connect is a framework for streaming data between Apache Kafka and external systems. It runs as a cluster of workers with a REST API (port 8083), and moves data through connector plugins that are supplied at deploy time via the connect-plugin resource.

Component	VM charm	K8s charm	Workload
Kafka Connect	`kafka-connect`	`kafka-connect-k8s`	charmed-kafka snap / OCI image

For instructions on deploying and using Kafka Connect through the API, see the Kafka Connect guide.

Connect integrators¶

Connect integrator charms are lightweight operators that package a specific connector plugin and integrate it with a Kafka Connect cluster, so you don’t have to build and attach plugin resources yourself. They are all built from the same template repository, canonical/template-connect-integrator.

Integrator	VM charm	K8s charm	Direction	Connector plugin
MySQL	`mysql-connect-integrator`	`mysql-connect-k8s-integrator`	Source and sink	Aiven JDBC connector
PostgreSQL	`postgresql-connect-integrator`	`postgresql-connect-k8s-integrator`	Source and sink	Aiven JDBC connector
MongoDB	`mongodb-connect-integrator`	`mongodb-connect-k8s-integrator`	Source and sink	Debezium MongoDB connector
OpenSearch	`opensearch-connect-integrator`	—	Sink	Aiven OpenSearch connector
S3	`s3-connect-integrator`	`s3-connect-k8s-integrator`	Sink	Aiven S3 sink connector
MirrorMaker 2.0	`mirrormaker-connect-integrator`	`mirrormaker-connect-k8s-integrator`	Replication	Built into Apache Kafka

Note

The Connect integrator charms are currently published to the edge risk level only, and are not yet covered by the stable revision compatibility guarantees described in the Compatibility section below. The OpenSearch integrator is available for machines only.

MirrorMaker 2.0 is a special case: it is not a source or sink connector but a replication mechanism built on Kafka Connect, used to migrate and replicate data between Kafka clusters. For how it works, see the MirrorMaker explanation, and for usage, the cluster migration and cluster replication guides.

Schema registry¶

Karapace is an open-source schema registry, originally developed by Aiven as a drop-in replacement for the Confluent Schema Registry. It stores and version-controls schemas for message serialisation, so producers and consumers can evolve their data formats safely.

Component	VM charm	K8s charm	Workload
Karapace	`karapace`	`karapace-k8s`	charmed-karapace snap

Source code: canonical/karapace-operator and canonical/karapace-k8s-operator. For managing schemas with Karapace, see the schemas and serialisation guide.

Cluster administration UI¶

Kafbat Kafka UI is an open-source web interface for browsing and administering Kafka clusters: topics, messages, consumer groups, brokers, and ACLs. It integrates with Charmed Apache Kafka, Charmed Apache Kafka Connect, and Charmed Karapace.

Component	VM charm	K8s charm	Workload
Kafka UI	`kafka-ui`	`kafka-ui-k8s`	charmed-kafka-ui snap

Source code: canonical/kafka-ui-operator. For usage, see the Kafka UI guide.

Client integration¶

The Data Integrator charm (source) is a workload-less operator that requests Kafka credentials and endpoints from a Charmed Apache Kafka cluster through the kafka_client integration, and exposes them to external (non-Juju) client applications via its get-credentials action. It is also used to enable client listeners on an otherwise idle cluster. For details, see the client connections guide.

Terraform modules¶

The Charmed Apache Kafka Terraform bundle deploys the whole component family with minimal configuration overhead. It is composed of one module per charm, each sourced from the charm’s own repository:

Module	Source	Charm
`broker`	kafka-operator	`kafka`
`controller`	kafka-operator	`kafka` (`roles=controller`)
`connect`	kafka-operator	`kafka-connect`
`karapace`	karapace-operator	`karapace`
`ui`	kafka-ui-operator	`kafka-ui`

The Data Integrator is deployed by the bundle as a plain juju_application resource rather than a module. For the full input and output reference of the bundle, see the Terraform module reference and the Terraform deployment guide.

How the components relate¶

The following diagram shows how the components connect to each other in a full deployment:

        flowchart TB
    subgraph kafka-model["Kafka Juju model"]
        direction TB

        broker["<b>kafka</b><br>roles=broker"]
        kraft["<b>kafka</b><br>roles=controller,balancer"]

        subgraph connect["Kafka Connect"]
            direction LR
            workers["<b>kafka-connect</b><br>Connect workers"]
            integrators["<b>Connect integrators</b><br>mysql · postgresql · mongodb<br>opensearch · s3 · mirrormaker"]
        end

        karapace["<b>karapace</b>"]
        ui["<b>kafka-ui</b>"]
    end

    subgraph clients[" "]
        direction LR
        client["<b>Client applications</b><br>producers · consumers"]
        di["<b>data-integrator</b>"]
    end

    broker -->|"kafka_client"| client
    broker -->|"kafka_client"| di
    kraft <-->|"peer_cluster"| broker
    workers <-->|"kafka_client"| broker
    integrators -->|"connect_client"| workers
    karapace <-->|"kafka_client"| broker
    ui -->|"kafka_client"| broker
    ui -->|"karapace_client"| karapace
    ui -->|"connect_client"| workers

Compatibility¶

The components above are released together per Apache Kafka major track (for example, 4/stable), so a deployment should use components from the same track: a 4/stable Kafka charm with a 4/stable Kafka Connect charm, and so on. The Kafka UI and Karapace charms are an exception: they are published to the latest/[risk] tracks (for example, latest/stable) rather than to per-major tracks.

The authoritative per-revision compatibility matrix — charm revisions, hardware architectures, Juju versions, and workload artefacts — is maintained in the release notes for each stable revision. The Connect integrator charms are published to edge only and are not covered by that matrix. For Juju versions, hardware requirements, and supported architectures, see the system requirements.