Enable tracing¶

This guide walks through enabling distributed tracing for your JAAS deployment.

JAAS ships a JAAS Traces Grafana dashboard that lists recent JIMM traces and lets you open any trace to inspect its span waterfall. Traces are collected by Tempo, so the dashboard is provisioned once JIMM is integrated with a Tempo instance.

What is traced¶

JIMM instruments the request path end to end. A single Juju API call typically produces a trace with a jimm.facade root span and nested child spans:

Span name	Created for
`jimm.facade`	Every Juju API facade call received over the Juju websocket RPC connection (root span)
`jimm.db`	Every database operation (create, query, update, delete, row, raw)
`jimm.openfga`	Every OpenFGA authorization check (with an `openfga.operation` attribute)
`jimm.juju-dial`	Establishing a connection to a managed Juju controller
`jimm.juju-websocket-dial`	Opening the websocket to a managed Juju controller
`jimm.juju-rpc`	RPC calls forwarded to a managed Juju controller (with facade, version and method attributes)
`jimm.model-proxy`	RPC messages proxied through to a managed model (with trace propagation to the controller)

This means traces appear for Juju API activity — juju CLI commands against the JIMM controller, dashboard sessions issuing Juju API calls, and any Juju client interaction — but not for plain HTTP requests such as the OAuth login redirect.

Note

On an idle deployment no traces are produced. Traces appear as soon as JIMM handles Juju API calls.

Prerequisites¶

A running JAAS deployment
A deployed COS stack including Tempo (coordinator and worker) with a configured storage backend

Integrate with Tempo¶

JIMM exposes a tracing integration endpoint:

JIMM endpoint	Interface	Provides
`tracing`	`tracing`	OTLP trace export endpoint and the JAAS Traces dashboard

To integrate JAAS with the Tempo coordinator of your COS stack — directly or through an application offer — follow the integration instructions in Integrate JAAS with the Canonical Observability Stack.

Once the relation settles:

JIMM receives the OTLP endpoint over the relation and starts exporting traces to Tempo.
Grafana receives the Tempo datasource and the JAAS Traces dashboard.

Sampling¶

JIMM samples traces according to the tracing-sample-ratio charm configuration, which defaults to 0.1 (10% of traces). To capture every trace — for example while testing or debugging — set the ratio to 1.0:

juju config jimm tracing-sample-ratio=1.0

Warning

Sampling all traces has a performance impact: every Juju API call produces a trace that must be exported to Tempo. Keep the ratio at 1.0 only while testing or debugging, and restore a lower value (for example the default 0.1) in production.

Verify¶

Open the Grafana UI of your COS deployment and log in. In the Grafana UI, open Dashboards and select JAAS Traces. The dashboard lists the most recent JIMM traces:

Column	Content
Trace ID	The trace identifier (clickable)
Start time	When the trace started
Name	Service and root span name
Duration	Total trace duration

Click a Trace ID to open the trace in Grafana Explore, where the full span waterfall is rendered.

If the list is empty, generate some Juju API activity (for example, run a juju command against the JIMM controller, or use the Juju dashboard to interact with a model) and refresh the dashboard.

Troubleshooting¶

The dashboard is missing: check that the tracing relation is active (juju status) and that Grafana received the Tempo datasource.
The dashboard is present but the trace list is empty: confirm there has been Juju API activity, that tracing-sample-ratio is not 0, and that the time range (top right of the dashboard) covers the activity.
Traces are missing spans: only Juju API facade calls and database operations are traced — see What is traced. Plain HTTP requests do not produce traces.

For the up-to-date definition of the dashboard, see the Grafana dashboards in the JIMM charm source.