Amazon, according to Amazon, aims to be "Earth's most customer-centric company." They do that by selling every product that can be sold online and more on their jungle-like marketplace. What's non-obvious about how they do business is their unparalleled bias towards experimentation.
Just to name a few successes that came through their rigorous test-and-learn process:
If that wasn't enough, here's an excerpt from Jeff Bezos' iconic 1997 Annual Letter to Shareholders:
"One area where I think we are especially distinctive is failure. I believe we are the best place in the world to fail (we have plenty of practice!), and failure and invention are inseparable twins. To invent you have to experiment, and if you know in advance that it’s going to work, it’s not an experiment.
Most large organizations embrace the idea of invention, but are not willing to suffer the string of failed experiments necessary to get there.
Outsized returns often come from betting against conventional wisdom, and conventional wisdom is usually right. Given a ten percent chance of a 100 times payoff, you should take that bet every time. But you’re still going to be wrong nine times out of ten.
We all know that if you swing for the fences, you’re going to strike out a lot, but you’re also going to hit some home runs. The difference between baseball and business, however, is that baseball has a truncated outcome distribution. When you swing, no matter how well you connect with the ball, the most runs you can get is four. In business, every once in a while, when you step up to the plate, you can score 1,000 runs.
This long-tailed distribution of returns is why it’s important to be bold. Big winners pay for so many experiments."
And a one-liner to drive it home: "Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.”
Instead of zooming in on a single experiment, consider the process by which Amazon runs experiments.
After wasting millions on tiny tests with no logic behind them, they built an idea vetting process:
Amazon ensures that all members of experiment groups understand statistics and have strong data-driven mindsets, because much like Booking.com, the highest-paid person's opinion is not the most important nor the final vote.
What's unique to Amazon's experiments is their typical process of idea generation. They reverse-engineer from customer experience, considering every step of interaction with their many interfaces while focusing on fostering long-term satisfaction.
That being said, they learn just as much from their failures as they do from their wins. If you haven't heard of A9, Auctions, or Endless it's because none of them worked out. Bezos claims they were all "me too" ideas, copycats of other companies. However, Auctions evolved into the idea for third-party sellers and Endless evolved into the Zappos acquisition.
Like fellow FAANG stocks Apple, Netflix, and Google, Amazon crushes its competition, consistently expanding deeper into the online retail world through experimentation. With $11.5 Billion in revenue, 19 Wiki-worthy subsidiaries, and over a million employees, they do fairly well, to say the least.
With current projects (that we know of) ranging from Economics to Robotics, Amazon is forging its way towards a data-driven future.
Even if your entire team is only Two Pizzas large, you can still apply Amazon's decision criteria to your experiments. Try focusing on reversible decisions that have a chance to "score 1,000 runs" with one swing of the bat.
