--- title: Overview | Apache Kafka description: Apache Kafka is a free, open source event streaming platform that enables you to continuously process data at web scale. url: https://canonical.com/data/kafka/what-is-kafka?format=md --- # What is Kafka? Apache Kafka is a free, open source event streaming platform that enables you to create queues for data buffering. Kafka is typically used to help build streaming data pipelines for distributed applications that continuously process data. It offers functionality for storing, processing and exposing event data streams. [Download the Kafka reference architecture guide](https://ubuntu.com/engage/kafka-reference-architecture) [Learn more about Apache Kafka ›](https://ubuntu.com/blog/what-is-apache-kafka) --- ## The benefits of Kafka Kafka provides critical functionality within modern distributed systems engineering: --- Produce and consume streams of event data --- Hold event data in the order in which it was produced --- Process continuous event data streams --- ## Why choose Kafka? * Clustered for resilience * Scale out for capacity * Broad ecosystem of tools --- ## Why do companies use Kafka? --- ### Proven solution Apache Kafka has been battle tested in real world deployments around the globe. ### Massive scalability Kafka is designed to run at web scale – with horizontal scaling capabilities built in. ### Large user community Kafka is a mature, actively developed project and has a vast user community. --- ## How do companies use Kafka? --- ### Streaming data You can use Kafka to buffer and distribute continuous data streams, for example to distribute web clickstream to downstream applications like databases, storage archives and reporting applications. --- ### Microservices Kafka is a popular choice for enabling microservices architectures, acting as a reliable channel for distributed, inter-process communications. --- ### Analytics With Kafka, you can integrate analytics processing directly into your data streams, enabling live insights and reporting. --- ## How does Kafka work? Kafka enables you to group events with a common theme into a logical structure called a topic. Topics are ordered sequences of event data records. Topics are usually replicated across Kafka broker servers in order to ensure event data durability and availability. Producers publish events to a given topic. Consumers subscribe to a topic and read the events that have been published to the topic. Many producers can publish events to a given topic. Likewise, many consumers can consume from a given topic. Kafka will track their read position in the sequence. Multiple consumer processes can be deployed to operate in a coordinated, parallel manner in order to scale reading from a topic by using Kafka's consumer groups feature. In addition to the core functionality, Kafka includes the Kafka Streams API, which enables users to develop data processing and analytics applications that run directly on the Kafka brokers. The Kafka ecosystem also includes Kafka Connect, an integration system for connecting all kinds of technologies to Kafka, from relational databases and other messaging technologies, to object storage systems. --- ## Feature breakdown --- ### Horizontally scalable Kafka clusters can be scaled to increase processing capacity by adding additional servers, a huge improvement over the vertical scalability limitations of other message brokers. --- ### Distributed processing Kafka workloads are distributed across many servers, which can massively improve performance by spreading concurrent client requests across the cluster. --- ### Durable Kafka is designed to replicate copies of your event data across the cluster, so that data loss is minimised when a server goes offline. --- ## Installing Kafka Kafka is a distributed system developed in Java, designed for computers running Ubuntu and other Linux distributions. You can use Charmed Apache Kafka to deploy a fully supported Kafka service on cloud servers, Kubernetes, or even on your laptop. [Try Charmed Kafka](https://canonical.com/data/kafka/docs/latest) --- ## Kafka security Apache Kafka includes mature information security controls like TLS wire encryption, authentication and authorisation and access logging. --- ## Charmed Apache Kafka Charmed Kafka delivers up to 10 years of support and security maintenance for Apache Kafka as an integrated, turnkey solution with advanced management features. --- ### Charmed Kafka Included in Ubuntu Pro + Support When you purchase an Ubuntu Pro + Support plan, you also get support for the full Charmed Kafka solution. --- * Up to 10 years of Kafka support per release track * 24/7 or weekday phone and ticket support * Up to 10 years of security maintenance for Kafka covering critical and high severity CVEs Charmed Kafka allows you to automate deployment and operation of Kafka at web scale in the environment of your choice – on the cloud or in your data centre. Supports deployment to most popular clouds or to CNCF conformant Kubernetes. [Learn more about Charmed Apache Kafka ›](https://canonical.com/data/kafka) --- ### Kafka OCI-compliant container image Included in Ubuntu Pro + Support Also included in Ubuntu Pro + Support, you get support for Canonical's OCI-compliant container image for Kafka in Docker Hub and Amazon ECR public – the slimmest, fastest, niftiest Kafka container image on the market, based on Ubuntu LTS. --- * Up to 10 years of support per release track * Same 24/7 or weekday phone and ticket support commitment * Same 10 years of security maintenance covering critical and high severity CVEs in the image [Try the OCI image ›](https://hub.docker.com/r/ubuntu/kafka) --- ### Kafka consultancy and support Advanced professional services for Kafka, when you need them Get help designing, planning and building and even operating a hyper automated production Kafka service that perfectly fits your needs, with Canonical's expert services. --- * Help with design and build of both production and non-production Kafka environments with Charmed Kafka * Managed services for Kafka in your cloud tenancy or data centre, backed by an SLA * Firefighting support with a Kafka operations expert, who works alongside your team when crisis hits [Contact us ›](https://canonical.com/contact-us) --- ## Learn more about Kafka Apache Kafka is more than just a buzzword in the tech community; it's a critical tool for data processing and management. Learn more in our webinar [Access our webinar ›](https://ubuntu.com/engage/kafka-introduction) --- ## Kafka resources * [Charmed Kafka reference architecture guide](https://ubuntu.com/engage/kafka-reference-architecture) Read the reference architecture guide for Charmed Kafka. * [Make better decisions with open source Big Data and AI solutions](https://ubuntu.com/engage/open-source-big-data-ai) Learn how to build a smarter enterprise with a secure, integrated open source stack. * [Migrating enterprise Kafka workloads from RHEL to the Ubuntu Platform](https://ubuntu.com/engage/migrate-kafka-workloads-from-rhel-to-ubuntu) Many organisations are considering moving core server workloads like Apache Kafka away from Red Hat Enterprise Linux. For those looking for an alternative, supported Linux distribution, Ubuntu is a compelling choice. Close ### Talk to our Apache Kafka solution architects --- Fill in this form and a member of our team will be in touch shortly.