File system paths

Paths differ because the VM charm installs the workload as a snap while the K8s charm runs it in an OCI workload container.

In the following table, we summarise some of the most relevant file paths used in Charmed Apache Kafka:

Environment Variable

Path

Description

Permission

BIN

/snap/charmed-kafka/current/opt/kafka

Binary files for the Charmed Apache Kafka distribution. Note that this is a read-only Squashfs file system.

read-only

CONF

/var/snap/charmed-kafka/current/etc/kafka/

Configuration files used by Apache Kafka daemon process. These files are generally written and managed by the charm.

Owned by _daemon_ Managed by charm

LOGS

/var/snap/charmed-kafka/common/var/log/kafka/

Application Logging files generated by the Apache Kafka daemon process. These files are written by the workload, but they may be read by other components to provide monitoring (for example, Grafana or other charms).

Owned and managed by _daemon_

DATA

/var/snap/charmed-kafka/common/var/lib/kafka/

Raw data stored persistently by Apache Kafka during its operations. The files are written and managed by Apache Kafka only.

Owned and managed by _daemon_

All of the environment variables ($CONF, $BIN, $LOGS and $DATA) are written to /etc/environment on the charm units, and as such are automatically available for all SSH sessions.

For example, to list the files and directories in the $LOGS directory on a particular unit, simply do:

juju ssh kafka/0 sudo -i 'ls $LOGS'

Variable

Path

Description

Permission

BIN

/opt/kafka

Apache Kafka binaries bundled in the charmed-kafka OCI image

read-only

CONF

/etc/kafka/

Configuration managed by the charm

managed by the charm

LOGS

/var/log/kafka/

Workload logs

written by the workload process

DATA

/var/lib/kafka/

Persistent message data

written by the workload process

Note

Unlike on VM, these variables are not exported as environment variables inside the container. Use the literal paths shown in the table.

The same layout is used for Cruise Control under /opt/cruise-control, /etc/cruise-control, /var/log/cruise-control, and /var/lib/cruise-control. Use the workload container when inspecting a path:

juju ssh --container kafka kafka/leader 'ls /var/log/kafka'

Configuration

  • $CONF/server.properties - the full configuration file for the broker and KRaft controller services

  • $CONF/client.properties - a minimal configuration file for making client connections to brokers with SASL authentication and SSL encryption

    • For internal administrator usage only

      • --command-config $CONF/client.properties is often used as an argument when running Kafka CLI commands either directly, or via the snap commands

  • $CONF/kraft-client.properties - the minimal configuration file for making client connections to KRaft controllers with SASL authentication and SSL encryption

    • For internal administrator usage only

  • $CONF/peer-keystore.p12 + $CONF/peer-truststore.jks - the Java keystore and truststore used for inter-broker and broker-controller SSL encryption

    • Passwords to the keystore and truststore are stored in Juju secrets

  • $CONF/client-keystore.p12 + $CONF/client-truststore.jks - the Java keystore and truststore used for client SSL encryption

    • Passwords to the keystore and truststore are stored in Juju secrets

  • $CONF/cruisecontrol.properties, $CONF/cruise_control_jaas.conf, and $CONF/capacityJBOD.json contain the Cruise Control configuration.

  • $CONF/jmx_prometheus.yaml configures the broker’s JMX exporter.

System logs

  • $LOGS/server.log - the broker service logs

    • Very useful for general debugging

  • $LOGS/kafka-authorizer.log - Kafka ACL authorisation events

    • Very useful for debugging authorisation failures

  • $LOGS/controller.log - the KRaft controller service logs

Apache Kafka binaries

  • $BIN/bin/*.sh - general bash scripts provided from upstream Apache Kafka, with utilities for managing and interacting with the cluster

    • On VM, these are typically exposed as snap commands; for example, kafka-topics.sh can be invoked with charmed-kafka.topics.

    • On K8s, run the scripts in the kafka workload container.

Message data and cluster metadata

  • $DATA/data/* - the data storage directory where the raw Apache Kafka message data is persisted to disk

    • Each Juju mounted JBOD storage directory will have an integer identifier matching a subdirectory in $DATA/data/

    • Find these directories with juju status --storage kafka | grep data/ on VM or juju status --storage kafka | grep data/ on K8s.