Metrics vs Traces vs Logs

Observability has three signals, and most self-inflicted pain — slow dashboards, cardinality blowups, runaway storage — comes from using the wrong one for the job. They are not interchangeable. Each answers a different question at a different cost.

Metrics — “is something wrong?”#

Numbers over time: request rate, error rate, p99 latency, queue depth. Cheap and aggregatable — a million requests to one endpoint collapse into a few series. This is what alerts fire on and what dashboards show.

The price of that cheapness: metrics are only cheap when their labels are bounded. Put an unbounded value (user id, raw URL) in a label and you get a cardinality explosion — because each unique value is a new series to store forever. Metrics are for bounded, aggregatable dimensions. Full stop.

Traces — “where is it wrong?”#

A trace follows one request across the system, as a tree of spans (each a unit of work). This is where high-cardinality detail belongs: a trace can carry the user id, the exact URL, the SQL text — because it’s one request, not an aggregate. When a metric tells you p99 latency spiked, a trace tells you it was a specific slow Mongo call. Traces are usually sampled (you don’t keep every one) because the full detail is expensive at volume. See debugging with SigNoz.

Logs — “what exactly happened?”#

The raw event stream — the highest fidelity, the highest volume. When a trace shows you which span errored, its logs show you the exception text and stack. Correlate logs to traces by trace id and you go from “this span failed” to “here’s the error” in one hop.

The rule that ties them together#

Metric is metric, trace is trace. Bounded, aggregatable → metric. Unbounded, per-request detail → trace or log. When you find high-cardinality data in a metric, it’s in the wrong signal.

This one rule prevents the two most common failures:

1. Cardinality blowups — an unbounded label sneaks into a metric. It belonged in a trace.
2. Slow dashboards — a panel computes an aggregate (like request rate) by scanning raw traces instead of reading a metric. Reading millions of spans to draw a line chart is orders of magnitude more expensive than reading a pre-aggregated metric. If a dashboard is slow, check whether it’s secretly querying traces.

Picking the right one, fast#

Start broad and cheap (metrics: is something wrong?), zoom to the request (traces: where?), then to the event (logs: what exactly?). That funnel is the fastest path through the debugging loop — and it keeps each signal doing the job it’s cheap at.