Throughout my work with product teams, I kept seeing the same pattern. Whenever the conversation turned to analytics, it almost always started the same way: Which events should we track? Which KPIs belong on the executive dashboard? What new funnel should we build? None of these questions are wrong. The problem is that they’re usually asked too early.

Before deciding what to measure, product teams should first decide what decisions analytics is supposed to support. In practice, however, the order is often reversed. Teams build increasingly sophisticated measurement systems, hoping those systems will eventually answer whatever questions arise later. Over time, the measurement system develops a logic of its own. Every new feature generates additional events, every experiment creates another report, and every quarter introduces new KPIs while almost none of the old ones disappear. Dashboards become remarkably good at describing the product—but surprisingly ineffective at helping teams decide what to do next.

Eventually, I started thinking about this pattern as the Dashboard Trap: the moment when analytics quietly stops being a decision-making tool and becomes an observation tool instead.

When Marketing Isn’t the Problem

I first noticed it while working with companies preparing to scale through paid acquisition. The conversations almost always began with marketing metrics — CAC, ROAS, acquisition channels, campaign performance — but they rarely ended there.

One team I worked with believed that rising customer acquisition costs were the primary reason growth had stalled. They had dashboards for every marketing metric imaginable and were already looking for new acquisition channels. Yet a closer look revealed a different problem: fewer than half of new users reached their first moment of value, and retention after the first week was far too weak to support additional marketing spend. The issue wasn’t acquisition—it was product readiness.

Once we looked beyond the advertising dashboards, it became clear that the real bottlenecks had very little to do with marketing. Users weren’t reaching their first moment of value, retention was too weak to justify additional acquisition, onboarding failed to communicate the product’s value, or the team still hadn’t identified a clear ideal customer profile.

In other words, the advertising campaigns weren’t failing because they had been optimized poorly. They were failing because the product underneath them wasn’t yet ready to scale. More traffic simply exposed existing product problems faster.

Visibility Isn’t the Same as Clarity

What surprised me most wasn’t the lack of data — it was the opposite. Most of these companies already had mature analytics stacks, detailed dashboards, and carefully defined KPIs. What they lacked wasn’t visibility into the product, but a shared understanding of which decisions those metrics were actually meant to support.

This observation echoes a point Marty Cagan

The more often I encountered this pattern, the more I found myself replacing one familiar question with another. Instead of asking, “What should we measure?”, I started asking, “What decision are we actually trying to make?” Eventually, I realized that the problem wasn’t the amount of data product teams collected. It started much earlier—with the first question they asked.

Decision-Led Analytics

Start With the Decision

That realization led me to a simple principle I now use in almost every product discussion:

Start with the decision, not the metric. I call this approach Decision-Led Analytics.

The idea is straightforward. Instead of asking “What should we measure?”, start by asking “What decision are we trying to make?” Once that decision is clear, it becomes much easier to identify which data actually matters.

A Different Conversation

Take onboarding as an example. Most teams immediately begin reviewing activation, completion rate, time-to-value, Week 1 retention, drop-off between steps, error rates, and dozens of other metrics that modern analytics platforms make easy to collect.

But imagine starting from a different place.

Instead of asking, “What should we measure?”, the team asks, “Should we redesign onboarding?”

Collect Evidence, Not Metrics

That simple change reframes the entire discussion. The objective is no longer to understand everything about user behavior; it’s to reduce enough uncertainty to make one specific product decision. Sometimes that requires only a handful of metrics. Sometimes it requires customer interviews, usability tests, or a small experiment. The point isn’t to collect less data—it’s to collect only the data that can change the decision

One Decision Before One Metric

This way of thinking builds on a principle introduced in Lean Analytics, where Alistair Croll and Benjamin Yoskovitz argue that companies should focus on the One Metric That Matters at each stage of growth instead of trying to optimize everything at once. I think the same logic applies one step earlier: before identifying the metric that matters, define the decision that matters. Once that decision is clear, the metrics usually become obvious.

Why This Matters More Than Ever

The Constraint Has Changed

The timing of this shift isn’t accidental.

For most of the history of software, the biggest constraint wasn’t deciding what to build—it was actually building it. Shipping a new feature required weeks or even months of engineering effort, so investing heavily in research, planning, and analytics before making a decision made perfect sense. The cost of getting it wrong was high.

Execution Is Becoming Cheap

Today, that equation has changed.

AI has dramatically reduced the cost of execution. A prototype that once required an entire team can often be built by a single developer in a matter of days. The technical barrier to shipping new ideas has never been lower.

As execution becomes cheaper, however, another constraint becomes more visible: decision quality.

Why Analytics Needs to Change

When building is no longer the bottleneck, choosing the right problem, the right feature, or the right customer becomes the real competitive advantage. Shipping the wrong thing is easier than ever—which also means it’s easier than ever to waste time building something that shouldn’t have been built in the first place.

That’s why I believe the role of analytics is changing. Its job is no longer to maximize visibility into everything happening inside a product. Its job is to reduce uncertainty just enough for a team to make the next decision with confidence.

The faster we become at building products, the more valuable disciplined decision-making becomes. In the AI era, the teams that win won’t necessarily be the ones that ship the fastest—they’ll be the ones that consistently decide what’s worth shipping.

Conclusion

Most product teams don’t have a data problem—they have a decision problem.

Over the past decade, we’ve become exceptionally good at measuring products. We know how to instrument every click, visualize every funnel, and monitor hundreds of KPIs in real time. But somewhere along the way, many teams started treating measurement as the goal rather than the tool.

That’s the Dashboard Trap.

The purpose of analytics has never been to explain everything that’s happening inside a product. Its purpose is to reduce uncertainty just enough for a team to make a better decision. Every dashboard, KPI, or tracking event should exist for one reason: to make a specific product decision easier.

That’s why I now try to ask a different question before adding another metric:

What decision will this information help us make?

If the answer is clear, the metric probably belongs in your analytics stack. If it isn’t, collecting more data is unlikely to change the outcome.

As AI continues to reduce the cost of building software, execution will become less of a competitive advantage. Decision quality will become more important than ever. The teams that win won’t necessarily be the ones with the biggest dashboards or the most sophisticated analytics. They’ll be the ones that consistently know which decisions deserve evidence—and which evidence is actually worth collecting.

In the end, the best dashboards don’t measure more.

They make the next product decision easier.