Why Most Validation Experiments Fail (And How to Fix Yours)

I always say validation is a founder or venture builder's full-time job. There is no end to validation until you have sold the company. But I feel validation doesn't get the importance and focus it deserves. Maybe it's being diluted in concepts like lean or growth, but at the end of the day, all are validation and validation is an experimentation journey.

Here's the problem: most people know they should run experiments. They've read The Lean Startup. They understand MVPs. They might have even run a few tests. But they're still running the wrong experiments at the wrong time, collecting weak evidence, and wondering why their validated ideas aren't working in the market.

The issue isn't that experiment libraries don't exist. There are plenty of them.

The issue is that people don't understand how to choose experiments, sequence them correctly, or recognize what constitutes strong evidence versus wishful thinking.

The Fundamental Mistake: Building First, Validating Never

Most people shoot directly into product. This has always been a problem, but with AI making it easier to build things, it's getting worse. Vibe coding and AI-assisted development have democratized creation. If you can dream it, you can build it. And that's exactly what people are doing, without ever validating whether anyone wants what they're building.

The biggest assumptions in any business are about the market, meaning the customers. When you test for the product too early, you make the very risky assumption that you're confident in what people will want and need. A shitty product is not an MVP. And more importantly, have you even tested for customers yet?

This isn't just about sequence. It's about creating a fog of uncertainty. When you build first and then try to get traction, you have no idea what's failing. Is it because people don't have the problem? Because they already have adequate solutions? Because your product is confusing? Because they can't get past sign-up? You've created a situation where you're running in all directions trying to figure out what's wrong, when you should have validated the market before writing a single line of code.

Building Things that Nobody Wants

The Three Core Problems Everyone Gets Wrong

After working with founders and corporate innovation teams, I see the same patterns repeating. There are three fundamental problems that cause experiments to fail:

Problem 1: Wrong Sequencing

You must always start with market validation before moving to product. There is no linear path from market to product, but these two facts are non-negotiable: start with market, and only move to product once you're fairly confident in what the value proposition should be based on validated learning.

The sequencing problem shows up everywhere. A founder builds an AI-powered personal finance coach, spends two weeks getting a working prototype with bank connections and spending analysis, tests it with friends who say "this is cool," and then can't figure out why they have no traction. They've created uncertainty. Is the problem that friends don't actually care about the value proposition? That they're already using something and don't want to switch? That the product is hard to use? There's no way to know because they skipped market validation entirely.

Or consider the corporate innovation team that interviews 25 quality control managers, documents their pain points, quantifies 15 hours per week spent on manual data entry, gets leadership approval for a $500K budget, builds an internal MVP, and watches adoption die. Two out of 15 managers use it regularly. What went wrong? They validated the problem but never tested whether managers were willing to fix it, whether the manual process served other crucial purposes, or whether managers would stick their heads out to try something new in a risk-averse culture.

Problem 2: Weak Evidence

Opinion rarely translates into behavior. A customer saying they would buy something rarely means they will buy it. This is extremely weak evidence, so weak that it doesn't even exist.

When testing for behavior, you should complete the experiment with a behavior simulation. Having a fake pay button. Asking for email addresses and follow-up information. Anything that causes the experiment audience to take action rather than just stating yes or no.

The mistake isn't just asking for opinions. It's treating interest as validation. Consider a founder building a B2B marketplace connecting freelance CFOs with early-stage startups. They interview 30 startup founders, 28 say they need fractional CFO help. They interview 15 freelance CFOs, all say finding clients is hard. They create a landing page, get 200 email signups. They survey those signups, 150 say they'd pay $2000-5000 per month.

This looks like validation. But what have they actually tested? Interviews got opinions, not behavior. Landing page signups show interest, maybe intent since people submitted email addresses, but not commitment. Survey responses about willingness to pay are hypothetical. None of this predicts whether people will actually engage and pay when the service exists.

The real test could be manually matching CFOs with startups and seeing if startups book consultations, pay for services, and book follow-up engagements. That's behavior. Everything else is noise.

Opinions ve Behavior

Problem 3: Context Translation Failures

Your problem is not automatically the market's problem. Corporate teams solve their own problems all the time and then assume the market wants the same solution. They validate internally, build a solution that works for them, and then fail when they try to sell it externally.

A manufacturing company builds an internal tool for quality control. It solves their problem. They adopt it across facilities. Then they try to sell it to other manufacturers and get nowhere. What they didn't test is whether other companies have the exact same problem in the exact same way, whether those companies have different constraints or contexts that change what the solution needs to be, whether the internal solution translates to different tech stacks and compliance requirements, and most importantly, whether other companies would actually pay for a solution versus just acknowledging they have the problem.

This is testing a single subject, solving your own problem without wondering if other people have that problem. It's a valid starting point, but once you decide to go to market, you need to go back to market testing. Don't assume your context translates.

How to Actually Think About Validation Experiments

The real work happens before you choose an experiment. You need to look at your idea, problem statement, opportunity, value proposition, or product and be super critical and super objective. List your biggest risks and assumptions first. Once the list is complete and ranked, take each assumption and turn it into an experiment brief: I am assuming this, in order to test this assumption I will do this experiment and expect this result. Usually this experiment brief dictates the experiment any way.

The bigger the potential failure, the more important it is to test for it first. This isn't a checklist. It's critical thinking about what could kill your business model.

The Market Validation Chain

Let's say you want to build a B2B SaaS tool for supply chain managers in manufacturing. You believe they spend 10+ hours per week manually reconciling inventory data across multiple systems.

Your assumptions might include: they have this problem, the problem is painful enough to change their workflow, current workarounds are inadequate, you can build a solution that integrates with their systems, they'd pay $500 per month, their company would approve the purchase.

The biggest assumption is whether they actually have this problem. But before moving to testing purchase behavior, you should quantify the problem in manhours or dollars ineffectively spent. Then test whether current solutions are truly inadequate. At this point you have a fair understanding of the dollar value of your value proposition and where you can create an edge in the competitive market. Then you move to a basic purchase experiment to see if managers will show interest through action.

For problem quantification, the simplest approach is interviewing someone in your target group with the right questions, having them show or tell you what they do. The output could be process diagrams. In a corporate environment, you might ask for screen captures while they work. Asking people to keep journals works too.

For testing whether alternatives are inadequate, you can interview users about their current solutions. Or you can test competitive solutions yourself against problems raised in earlier tests. Create a matrix of problems and features for each competitive solution. Understand recurring features you might need, differentiating features you want to compete on, and unmet needs where you might create an edge.

Validation Progression

Interviews Are the Workhorse

If you master interviews, you will use them way more than you think. They're not weak evidence when done correctly. A strong interviewer is well-prepared. She knows the objective of the experiment. She has a list of questions that have been peer-reviewed. She has prepared any artifacts needed to enhance the interview. During the interview she speaks way less and listens way more, takes notes constantly, and never steers away from the objective. Steering away often shows up as defending prior assumptions or switching to pitch mode when a small opportunity appears. After the interview, she knows how to record learnings and quantify qualitative data.

The difference between a weak interview and a strong one is the difference between "Would you use a tool that reconciles inventory data?" and "Show me the last time you did this. Walk me through your screen. What was frustrating?" One is asking for an opinion about a hypothetical. The other is observing concrete behavior and identifying actual pain points.

When to Move From Cheap to Expensive Experiments

Use the cheapest experiment that can invalidate your current riskiest assumption. Only move to more expensive experiments when cheaper ones have validated enough that the next risk becomes worth the investment.

Interviews are low cost but can generate relatively strong evidence on problem existence and quantification when done well. Fake door tests or landing pages are medium cost and can provide medium-strong evidence on interest. Working prototypes with fake backends are higher cost with stronger evidence on behavior. MVPs with real functionality are the highest cost with the strongest evidence.

Naturally, experiments become more complex and evidence becomes more quantifiable as you move down the risk ladder. Early stage assumptions are broad: "Supply chain managers have inventory reconciliation problems." You're running exploratory experiments like interviews to understand the problem space, looking to invalidate big chunks of your idea quickly.

Mid-stage assumptions get specific: "Supply chain managers would interrupt their workflow to use our solution at least 3 times per week." You're testing specific behaviors with fake door tests, email signups, behavioral simulations. You're validating specific value proposition elements.

Later stage assumptions are granular: "Supply chain managers will complete onboarding and integrate our tool within 2 weeks." You're running higher fidelity experiments with actual product usage and pilot programs. You're validating operational and scaling assumptions.

The Corporate Innovator's Challenge

Corporate innovators face an additional layer of complexity. More often than not, they have corporate and shareholder pressure to deliver results, not learnings. They're less experienced in entrepreneurial approaches and need to balance validation rigor with corporate pressure.

My suggestion: use experiments that are slightly more complex than strictly necessary, so you can mark tasks complete in internal project management tools. This isn't about inflating costs unnecessarily. It's about understanding that corporate startups don't need to be as fast as independent startups. They have more resources and job security, so slightly higher experiment costs are acceptable trade-offs for maintaining internal support.

The guidance is to run the same rigorous experiments but make artifacts look more professional so you can show progress, choose experiments that naturally produce demonstrable deliverables like prototypes or research reports, and build demonstration value into your experiments so you can present learnings to stakeholders.

The risk is spending more than the evidence warrants or falling into the building-too-early trap with better justification. But if you stay true to experimental rigor while excelling at reporting to manage stakeholders, you can navigate this tension.

Corporate innovators share the same risks as founders when it comes to building too early or relying on weak evidence. The difference is they need to manage internal stakeholders while running experiments. That's an extra skill, not a different approach to validation.

How to Use an Experiment Library

Once you've done the work of identifying and ranking your assumptions, an experiment library becomes useful. That why I have one right here 😊. It helps you match the right experiment to your assumption and understand sequencing logic.

The library isn't meant to help you pick assumptions. That's the hard thinking work you need to do yourself. Be super critical about what could kill your business model. Rank those risks. Turn them into testable assumptions. Then find experiments that can invalidate those assumptions cheaply before you invest in building.

Look for experiments that match your current stage and resources. Understand the "do this before, with, and after" relationships to build logical experiment roadmaps. Filter by what you're testing: market assumptions, product assumptions, business model assumptions.

But don't browse the library first hoping to find inspiration about what to test. Start with critical thinking about your biggest risks. The library helps you execute that thinking, it doesn't replace it.

What Actually Matters

AI has made building faster and easier. Vibe coding, no-code tools, and AI assistants have democratized creation. If you can dream it, you can build it. That's why the discipline of validation matters more than ever.

Navigation is the Key

When implementation barriers drop, the temptation to build first becomes stronger. But building first creates fog. You can't isolate what's failing. You end up running experiments in all directions trying to figure out what assumption was wrong when you should have tested those assumptions before writing code.

The fundamentals haven't changed. Start with market validation. Test for behavior, not opinions. Don't assume your context translates to others. Move from cheap experiments to expensive ones as evidence accumulates. These principles still hold.

What's different is that you need more discipline to follow them. When you can build a working prototype in two weeks, the pull to skip validation becomes almost irresistible. When AI can help you create a landing page, mockups, even functional code, you feel productive. But productive building without validation is just faster failure.

Validation is your full-time job until you've sold the company. Maybe it doesn't get the respect it deserves because it's been diluted into concepts like lean or growth. But at the end of the day, it's all validation. And validation is an experimentation journey.

The question isn't whether you should run experiments. It's whether you're running the right experiments at the right time, collecting evidence that actually predicts behavior, and being honest about what you've validated versus what you've assumed.

Most people get this wrong. But if you start with critical thinking about your biggest risks, test for behavior instead of collecting opinions, and maintain the discipline to validate before building, you'll have an advantage that compounds over time.

The experiments matter. But the thinking behind them matters more.