Six startup validation stories, fact-checked against the sources

We checked six famous validation stories against first-hand sources. Some numbers were wrong, two endings were wrong, and the real lessons are better.

Idea Validation

Rishabh Pandey

·
Share:
Six startup validation stories, fact-checked against the sources

We published the first version of this post in 2021. It told six famous stories of startups that validated their idea before building, and it had no sources. When we came back to update it, we checked every claim against first-hand sources: the companies' own announcements, the founders' own posts and contemporary reporting. Several numbers were wrong. Two endings were simply false: Dropbox was never bought by Microsoft, and we could find no source for a $200 WhatsApp demo video.

Below, each story is told as the sources tell it, with what we got wrong and what the real lesson is. These stories get retold in thousands of articles, usually with the same errors, so it's worth knowing which parts hold up.

Dropbox: the demo video was real, the ending wasn't

What we said in 2021: Houston set up a website, emailed sign-ups, demoed the product over dinner, reached 10 million users in a year and sold to Microsoft for $8 billion.

What the sources say: On 4 April 2007 Drew Houston posted "My YC app: Dropbox - Throw away your USB drive" to Hacker News, linking a short screencast of the product (Hacker News). In a 2011 interview with TechCrunch, he described a second video, posted to Digg during the private beta about a year later and full of in-jokes for that audience. The team expected 15,000 sign-ups at most; 75,000 people joined the waiting list within 24 hours (TechCrunch).

Dropbox was never acquired. It listed on Nasdaq in March 2018 at $21 a share (Dropbox via Business Wire).

The real lesson: the video wasn't a fake product. The software worked; the video showed it to people who couldn't try it yet. That's a cheap, honest test: it measures demand for a specific thing without promising something that doesn't exist.

Instagram: the validation came from a different app

What we said: the founders listed four features, built a prototype, got feedback from friends, simplified it, and had 40 million users when Facebook bought them.

What the sources say: Kevin Systrom and Mike Krieger first built Burbn, a check-in app where users could also post plans and photos. TechCrunch's 2012 account says beta users gravitated to sharing photos, so the founders scrapped most of Burbn and built a photo app that sat, in Systrom's words, between Hipstamatic and Facebook (TechCrunch). Instagram launched on 6 October 2010 and 25,000 people signed up on the first day. A month before the acquisition it had 27 million registered users, not 40 million, and about a dozen employees. Facebook announced the purchase on 9 April 2012 for about $1 billion in cash and shares (Meta).

The real lesson: the best validation data often comes from the product you already have. Burbn's users told the founders what to build by what they did, not by what they said. If you have any product with users, look at which feature they actually use.

Airbnb: three guests, not a party business

What we said: the founders rented air mattresses mostly to people holding small parties, ran the business for three months, then built a website.

What the sources say: Airbnb's own account is short: it "was born in 2007 when two hosts welcomed three guests to their San Francisco home", and the company was first called Airbed & Breakfast (Airbnb). There's no first-hand source for the party story; we got it wrong.

The real lesson: the first test was tiny and real. Three paying guests is a small number, but it's three strangers who paid to sleep in someone's home, which was exactly the behaviour everyone assumed people wouldn't accept. Validate the riskiest assumption first, even with a sample of three.

WhatsApp: no video, and a different user count

What we said: Jan Koum and Brian Acton made a demo video in an hour for $200, collected invite requests, and had 700 million users at the Facebook deal for $19 billion.

What the sources say: we couldn't find any first-hand source for the video or the invite list, so we've removed them. The deal is documented: on 19 February 2014 Facebook announced it would buy WhatsApp for about $16 billion in cash and shares, plus about $3 billion in restricted stock for employees, which is where the $19 billion figure comes from. At the time, more than 450 million people used WhatsApp each month, and 70% of them were active on a given day (Meta). The 700 million figure came later.

The real lesson: we don't have a documented validation story for WhatsApp, and we'd rather say so than repeat an invented one. The usage figure in the announcement is the kind of evidence that matters: most users came back every day.

Dollar Shave Club: the video sold, then the site fell over

What we said: the founders built a $1,000 website with a video and got 2,000 orders in 48 hours.

What the sources say: According to Entrepreneur's 2017 profile, Michael Dubin posted the company's launch video at 6 a.m. Pacific on 6 March 2012; it cost $4,500 to make, and the company took 12,000 orders that day. Unilever completed its purchase of the company for $1 billion on 19 July 2016 (Entrepreneur). Other accounts, such as Wikipedia's, give the 12,000 orders over 48 hours, so the exact window varies, but every version is far above our 2,000.

The real lesson: the product and the subscription were ready before the video went out; the video tested the message and the price. A launch that sells out also shows you what breaks first, which for most small companies is fulfilment and support, not the website.

LinkedIn: the ending is right, the middle isn't sourced

What we said: Reid Hoffman ran surveys, interviews and user tests, spent five months building, and Microsoft bought LinkedIn for $26 billion.

What the sources say: Microsoft announced on 13 June 2016 that it would buy LinkedIn for $196 a share in cash, about $26.2 billion, when LinkedIn had more than 433 million members (Microsoft). We couldn't find a first-hand source for the surveys and the five months, so we've removed them. Hoffman's own advice on launching is the opposite of a long research phase: his best-known line is that if you're not embarrassed by your first version, you launched too late. We cover what he actually meant in when to launch an MVP.

What the six have in common

Once the invented details are gone, the stories are shorter and more useful:

  1. Each test was real. Real software in a video (Dropbox), a real product with users (Burbn), real guests (Airbnb), real orders (Dollar Shave Club). None was a survey asking "would you use this?".
  2. The evidence was behaviour, not opinion: sign-ups, usage, payment.
  3. The numbers were small at first. Three guests. A screencast on a forum. The large numbers came later and are what everyone remembers.

How to check a startup story before you use it

Because these stories are retold so often, here is the routine we now follow before quoting one:

  • Find the first-hand source: the founder's own post, the company's announcement, or an interview from the time. If every source cites another article, treat the story as unverified.
  • Check the ending against a filing or announcement. Acquisitions and listings have press releases with dates and prices.
  • Expect numbers to drift. Dollar Shave Club's 12,000 orders appear as one day in one account and 48 hours in another. Use the earliest first-hand figure and say which one you used.
  • Remove what you can't source. A shorter true story is better than a longer invented one.

To design your own test, our idea validation guide covers what desk research can and can't tell you, and what a cheap market test can. If you run a survey as part of it, the sample size calculator shows how many answers you need before a percentage means anything.

Sources

Share:

Building a software product?

Talk it through with us on a free 30-minute call, or email [email protected]. You can also try our free tools.

Book a call