In the previous post I described the Observability Pain Cycle. Everything is stored up front, only a small part of it gets used, and the one thing that ever triggers a correction is the pain of a high bill. The correction is a one-time trimming exercise, the volume silently grows back, and the cycle starts over. The party that knows best what is valuable, you, has little say in that loop.

This post is about the opposite loop. It runs on the same architecture: the same workloads, the same agent, the same dashboards and alerts. Only two defaults are flipped. The result is a system where you decide what gets stored, and the cost follows that decision instead of preceding it. I call it the Observability Value Cycle.

Two defaults, flipped

The Pain Cycle rests on two defaults. The agent forwards everything unless somebody tells it otherwise. And whatever happens to land in the store dictates what you can ask of it later. Both favor the vendor, and neither is a law of nature.

The Value Cycle turns them around:

This may sound like a lot of declaring, but most of the declarations already exist. A dashboard panel is a query, and the query says exactly which series it needs. The same goes for an alert rule. The things that consume telemetry already describe the demand, and the platform can read it straight out of them. For everything else (data you want to keep for future investigations, for an audit, or simply on a hunch) you write an explicit rule.

How it works

The boxes are the same as in the Pain Cycle diagram. What changed is shown in green.

The Observability Value Cycle
Value drives the telemetry filtering.
Workloads apps & services Agent Filter forward NOTHING by default Observability Platform Store on Demand Demands What brings value? drives $$$$$ Store demanded telemetry Dashboards fast queries Alerts
Telemetry data flow Control flow

This is how the story unfolds this time:

The shape of the cost

In the Pain Cycle, the signal that triggers a correction is the invoice, and it is late and coarse. In the Value Cycle the signal is the demand, and it has the opposite properties. It is early: the decision is made before the telemetry is ingested, so the waste is never paid for. And it is fine-grained: it is made per panel, per alert, per rule, by the person who knows why the data is needed.

Cost Time Collect everything initially High bill 😡 Reactive pruning 😐 Pain Cycle Near-zero start Demand-driven growth 😊 Value Cycle

The shape of the curve follows from that. The cost starts near zero and grows only when the demand grows. When a team retires a dashboard or a service gets decommissioned, it goes down again. There is no spike at the start and no cleanup project in the middle, because a pile of unused telemetry never builds up.

How big the gap is depends on how much of your telemetry you actually use. It is biggest at the beginning, when the Pain Cycle already stores everything and uses nothing. But a gap remains even against a carefully pruned setup, because pruning is a snapshot that starts to drift the next day, while the demand is always current. And a smaller store brings a benefit that never shows on the invoice: queries run over the data you care about, not over the haystack around it.

The obvious objections

“What about the signal the next incident will need?” This is the honest core of the “you never know” argument, and the Value Cycle has to answer it. It does so in two ways. The incident that is happening right now is covered by the window at the source: everything the agent saw recently is there in full fidelity, demanded or not, and you can query it. And the data you already know you want for later investigations is a demand like any other. You declare it, it gets stored, and it comes with a visible price. What you give up is long history for signals nobody ever asked for. That is a real trade-off. But you make the same trade-off in the Pain Cycle every time you sample, drop, or shorten retention after a bill, only blindly and under pressure.

“So it is an allowlist.” It is not. A hand-maintained allowlist is the Pain Cycle’s drop filter with the sign reversed: somebody guesses what is needed, writes it down, and the list is outdated a month later. Here nobody maintains a list. It is derived from the dashboards, alerts, and rules that exist right now, and recomputed whenever one of them changes. And you never choose blindly, because everything that exists is visible at the source.

“So it is sampling.” Sampling and the other volume controls decide how much data to keep. A demand decides which data to keep. A demanded series is stored in full, and an undemanded one is not stored at all.

“It means re-instrumenting everything.” It does not. The workloads emit what they emit today, over the same protocols. Dashboards and alerts are written in the same query languages. The only thing that moves is the decision about what to store, and it moves from the vendor’s defaults to you.

Conclusion

The Observability Value Cycle is the Pain Cycle with two defaults flipped: forward nothing unless it is demanded, and let the demand, not the stored data, drive the loop. The architecture stays the same. What changes is who is in control. You decide which questions your observability should answer, the system stores what those questions need, and the bill becomes the sum of your own decisions.

None of this requires exotic technology. So why does it still look unusual? Because it is not a feature that can be added on top of a store-everything platform. It is a different way of thinking about telemetry. And our industry would not be the first one to go through such a change. Manufacturing went through it decades ago, when it learned that producing more than anybody asked for is not a safety margin but the most expensive kind of waste. That is the topic of the next post.

← All posts