Research · Supporter perk
What we decided to trust
Qualitative findings from thirteen interviews with founders, product leaders, and engineers on how they decide what to believe in their product data.
Final edit · 13 September 2026 · Research by A Founder's Friend · This research was sponsored by Skene.ai
The opening — "What, but rarely why"
Every founder we spoke to could tell us, to the datapoint, what their users did last week. Almost none of them trusted the data to tell them why. That gap — between the number on the dashboard and the reason behind it — turned out to be the thing thirteen builders circled back to: a consumer-app CEO, a solo AI founder, a fifteen-year product veteran, a London grocery-delivery engineering manager running an army of agents, a Finnish CTO rebuilding after his first product died, and a healthcare product officer who is not allowed to watch his users at all. Everyone has different roles and approaches to product work, but is united by being product builders.
Being data-driven has new elements to it in today's AI-native environment. You can prompt an application into existence in an afternoon; you cannot prompt the understanding of why someone abandoned it without a discussion with that someone. And the faster we build, the more quietly the measurement drifts out from under us: a feature ships, the tracking doesn't follow, and a decision gets made on a number that stopped being true three releases ago. In some environments, this loop happens fully autonomously, wasting credits and energy.
So we asked people who build for a living a deceptively simple question: how do you decide what to trust in your product data — and when does a bad number become a decision that costs you? Their answers are below.
The short version: the builders winning this aren't the ones with the most data. They're the ones who know exactly which number to distrust, and when intuition beats access to data.
The headline finding
Data decides the small, frequent, reversible calls. Everything else still goes to a person.
The line falls in the same place in almost every conversation. Where a decision is high-frequency, low-stakes, and easily undone, participants were happy to let the numbers run it — and increasingly happy to let an agent run it on their behalf. The moment a decision becomes ambiguous, expensive, or hard to reverse, every single participant reached for a human. Not because the data was wrong. Because nobody wanted to be the person who couldn't explain it afterwards.
Data always tells you what's going on. It never tells you why.
— Maria Petrova · Fractional CPO
The argument against this finding is that you're gathering more context from your users than what can be tracked by user behaviour in your digital product, such as written user feedback, customer interviews, or comments about your product online, and calling all of that "product data" means we're getting lost in the conceptual meaning of things before being able to debate the actual statements made by individual interviewees.
This research therefore includes suggestions for mental models for thinking about product data, and not only the findings from the discussions.
Continues below
Part 1 — The ten themes
- 01Data tells you what, never why
- 02Data validates decisions more often than it makes them — and where that flips
- 03The closer you can get to the user, the less you need the aggregate
- 04AI builds in hours; instrumentation is the afterthought
- 05The real danger is the silent decision
- 06AI can assist; it never gets the wheel
- 07One metric, five different answers — and nobody owns it
- 08One number, several meanings
- 09Sometimes there is no data at all — and that is not always a failure
- 10Building stopped being the bottleneck. Deciding what to build became it.
The full themes with the interview quotes, the seven-class data framework, the three registers of "data-driven", the four-phase decision model, the practice notes and the method are a supporter perk.
Checking your access…