We often copy what “worked” but rarely ask what “failed” and why.
In product teams, just like in life, we love success stories. We borrow onboarding flows from top startups, replicate features from fast-growing apps, and chase strategies that “worked for X.” But we rarely pause to examine what didn’t work, and what we can learn from it.
That’s the danger of survivorship bias. We focus on the visible winners, forgetting the quiet failures that never made headlines. You’ll hear people say, “So-and-so dropped out of school to build a billion-dollar company,” and use that as motivation to do the same. But for every one success like that, there are dozens, maybe hundreds who tried and failed. We just don’t see them.
Why?
Because survivorship bias hides them from view.
Why We Keep Falling for It
Survivorship bias persists because it’s comfortable.
Success stories are clean. They give us a narrative we can follow, a sense that if we do the same things, we’ll get the same results. They simplify complexity into something actionable.
Failure, on the other hand, is messy. It’s harder to access, harder to measure, and often undocumented. Most teams don’t always write detailed reports about what didn’t work. When experiments fail, they get quietly buried. If a feature becomes unsuccessful, it just quietly disappears.
The consequence for this is that product teams default to what’s visible, not what’s complete. And over time, this creates a distorted view of reality, one where success looks repeatable, but rarely is.
The War Story That Changed the Question
There’s a well-known example from World War II, when the United States was fighting the Axis powers particularly Nazi Germany.
American warplanes were being shot down at an alarming rate over enemy territory. The United States Army Air Forces needed a way to keep more planes and pilots alive.
So they started studying the aircrafts that made it back.
Bullet holes were everywhere like the wings, tail, fuselage. The conclusion seemed obvious, reinforce those areas with bullet holes.
But there was a problem.
They were only looking at the planes that survived. What they weren’t seeing were the planes that didn’t make it back, the ones hit in critical areas like the engine or cockpit. Those aircraft didn’t survive long enough to be studied. The planes that returned could afford to take hits on the wings or fuselage. The others couldn’t.
Once that insight was understood, the strategy changed. Instead of reinforcing the most damaged areas, they reinforced the areas that showed no damage.
And guess what? Yeah you guessed right.
Survival rates improved.
What This Really Means for Product Teams
The breakthrough in that story wasn’t just about where to add armor. It was about reframing the question.
Instead of asking:
“What’s working?”
They asked:
“What are we not seeing?”
That shift from visible data to missing data is where real insight lives. In product work, this might look like:
Which users dropped off before activation?
Which features were removed quietly?
Which experiments never made it into reports?
Which ideas looked promising but failed in execution?
The absence of data is often more important than what’s in front of you.
The Invisible Graveyard Behind Every Product
Every successful product has a graveyard behind it. Features that never shipped. Experiments that failed silently. Growth tactics that looked promising in strategy decks but confused users in reality.
We rarely document these losses.
So new team members come in, see what exists today, and assume it’s the result of a clean, linear path. They don’t see the iterations, the missteps, or the decisions that didn’t survive.
And because of that, teams repeat mistakes, not out of carelessness, but because the history of failure is missing.
The Cost of Ignoring Failure
When teams rely only on visible success, they don’t just miss insight - they introduce risk.
Roadmaps become reactive instead of intentional. Features are shipped based on imitation, not understanding. Experiments are run without clear hypotheses, because the assumption is “it already worked somewhere.”
The result isn’t just failed features. It’s wasted cycles, confused users, and slower learning.
“Success tells one half of the story, failure tells you the other half - and the two halves are the only way to see the truth.”
When Copying Goes Wrong
Consider a team that implements aggressive caching because high-performing systems rely on it for speed. On paper, it should reduce latency and improve performance.
They deploy it and users start seeing stale or inconsistent data. Bugs become harder to trace and the cascading effect is that trust in the system drops.
What went wrong?
The teams they copied had carefully designed cache invalidation strategies and data consistency models.
The new team copied the solution, but not the complexity behind it.
Same optimization. Missing constraints. Different outcome.
This is how survivorship bias plays out in real product decisions not as obvious mistakes, but as reasonable decisions made on incomplete information.
From First-Order to Second-Order Thinking
First-order thinking says:
“This worked, let’s do it.”
Second-order thinking asks:
Why did it work in that context?
What conditions made it possible?
Do those conditions exist here?
What happens if they don’t?
Survivorship bias thrives in first-order thinking. And if there’s anything I've learnt so far, it is that better product decisions come from going one layer deeper.
How to Apply This in Your Work
The next time you’re about to copy a feature, adopt a “best practice,” or reproduce strategy, pause and ask.
What problem was this actually solving?
What made it work for them?
What are we assuming about our users?
What similar attempts have failed and why?
What are we not seeing?
The idea isn’t to avoid learning from successful products. It’s to understand them well enough to know when their success doesn’t apply to you.
Where This Leaves Us
There’s nothing wrong with chasing what’s visible. But better product teams go further. They question what’s missing, not just what’s obvious. More importantly we should ask what didn’t survive and why.
Because success tells you one half of the story, failure tells you the other half, and the two halves are the only way to see the truth.
Don’t just ask who won, also ask who tried and disappeared, there are lessons there too.