B2B Engineering Insights & Architectural Teardowns

Platform engineering metrics without telemetry

Platform engineering metrics define the direction of platform development. Without them, the team cannot demonstrate value or manage evolution.

The problem does not manifest immediately — until the moment the platform ceases to be “obviously useful.” In many organizations, platform engineering teams lack basic metrics and baselines. This makes it impossible to assess progress. The situation resembles a system without telemetry (observability): as long as everything works, there are no questions, but when degradation occurs, there is no reference point for analysis. As a result, it is difficult to explain the value of the platform to both engineers and the business.

The absence of platform engineering metrics impacts several levels simultaneously. The team does not understand whether it is improving the developer experience. Users do not see transparency. Management does not receive signals for decision-making. In such a configuration, the platform turns into a “black box,” and any changes appear intuitive rather than managed.

As a starting point, it is suggested to use a narrow product — Unified Secrets Manager for Kubernetes. This is an API service that allows developers and AI agents to perform CRUD operations with secrets. The narrow scope here is a conscious choice. It reduces complexity and allows for faster construction of a measurable model. This is a compromise: less coverage, but higher accuracy of observations.

The solution is built around a scorecard approach. Metrics are grouped into several categories. The initial example mentions four sections, but their composition is not detailed. This is an important point: there is no universal set. The team must adapt the structure to its product and context. Nevertheless, the very idea of a scorecard provides a framework that helps avoid a chaotic set of metrics.

From an engineering perspective, the important aspect here is not the number of metrics, but their interconnectedness. A good model should answer three questions:

  • Is the platform being used?
  • How reliable is it?
  • Does it simplify users’ work?

If the metrics are not related to these questions, they quickly lose practical value. This is a typical mistake: collecting data for the sake of data.

Implementation begins at the API level. Unified Secrets Manager already provides a control point — all operations go through it. This simplifies telemetry collection: requests, errors, and latencies can be recorded. However, challenges arise in interpretation. For example, an increase in the number of requests can indicate either success (more users) or a problem (excessive retries or inefficient clients).

Another nuance is working with Kubernetes secrets. It is important to consider the context of use. The same pattern may be normal for one team and an anti-pattern for another. Therefore, metrics without segmentation quickly lose meaning. This imposes additional requirements on the data model.

Results are not provided in the original material. There are no specific metrics or numerical improvements. This is a limitation, but at the same time a signal: the task is not to immediately achieve a perfect measurement system. It is important to create the first version of the baseline. Even an incomplete model is better than its absence.

The practical effect of this approach is the emergence of a common language. The team can discuss the platform through data rather than feelings. This reduces uncertainty and helps establish priorities. Over time, the scorecard can be evolved by adding new dimensions and refining existing ones.

In the industry, such approaches have long been discussed, but often face implementation challenges. The main reason is the attempt to immediately build an “ideal” metric system. The example with Unified Secrets Manager shows a more pragmatic path: start with a narrow case, establish a baseline, and gradually develop the model.

Read

×

🚀 Deploy the Blocks

Controls: ← → to move, ↑ to rotate, ↓ to drop.
Mobile: use buttons below.