Is it possible to ‘simulate’ a product or business idea that only exists in your head right now?
What if we could understand exactly how customers would react to a hypothetical product — even down to the nitty gritty details of what your CAC (customer acquisition cost) would be, your ROAS (return on ad spend), and even which ad would work best for any given audience?
If this was possible, wouldn’t it allow us to rapidly asses product-market fit without even investing the ~$50–100k required to design, manufacture and ship a product? Yes, it would. It would be kind of like a cheat code, letting you take a peek into the future.
About a year ago, my team and I started to work with several well-capitalized entrepreneurs who were launching many different brands and products simultaneously — a business model referred to as a ‘venture studio’, which is when a single entity both founds and funds their own venture ambitions. This is in stark contrast to the ‘venture capital’ model, where one entity founds the idea and the other entity funds the idea.
Entrepreneurs running venture studios have a ton of advantages that skew the odds heavily in their favor — they have experience launching successful products before, they have capital, they have a reliable network of tested vendor relationships, and they have proven instinct.
Their biggest challenge actually has nothing to do with the typical challenges most founders face (funding, team building, ops), but instead has to do entirely with time. They have a lot of ideas, and not enough time to execute against all of them. Execution against a product idea is at least a 6–9 month process that requires product design, manufacturing/logistics negotiations, creative/web/content production, and launch campaign planning/deployment. All this, with only a 20% chance of successfully reaching product-market fit.
This is where we started playing with the idea of combining a design practice called Fake Doors and a marketing practice called Multivariate Testing (MVT). After much trial & error, we discovered that by combining the two principles in a strict methodology, we could successfully ‘simulate’ hundreds of product launch scenarios and tell entrepreneurs which products were worth launching and how (before any product even exists).
I’m going to peel back the curtain and show you how we do it for venture studios, how it works in a 3 step process, and how you can apply it to your product idea no matter who you are.
When I say ‘multivariate testing,’ almost everybody responds “Oh, that’s like A/B testing but on steroids.” In reality, the two are completely different practices that have almost nothing in common.
In an A/B test, you compare different ads that are run under the same circumstances. In a multivariate test, you identify variables (background image, copy, etc.), create multiple options (aka variants) for each variable group, and then you test every permutation (or, every possible combination of variants). This allows you to isolate and measure the impact of individual creative elements, empowering you to understand why your best ads work well (rather than just knowing which ad is the best one).
Multivariate tests are built in grids, as demonstrated above. They can range in size from only a few ads (the smallest possible grid would be 4 ads), or they can be hundreds/thousands.
Let’s say you’re launching a new juice product — there are a lot of variables that we all know would impact the success of this product (like flavors, packaging, color, branding, messaging, fonts, etc.). If we have a designer mock up these elements and combine them all together, we might get something that looks like this:
Above, you see a rendering of a hypothetical juice product, where we create an ad that represents every permutation of elements like colors, backgrounds, messaging, and layout. Below is another example for a dress company.
By doing this, we can ‘simulate’ hundreds of product-market fit scenarios at the top-of-funnel, allowing us to understand exactly which combination of elements will be the one that the market demands the most for our hypothetical product. This is critical because creative influences over 50% of sales outcomes in advertising, so it’s essential we use data to guide our creative instead of guessing or tastemaking.
Building a multivariate test can be a lot of work depending on how fancy you want to get . The bigger the multivariate test, the better your results will be — luckily, Marpipe can help you automate this entire process.
Fake doors is a very effective and common practice in the UX/UI community — some perceive it as evil or deceptive, but that all depends on how you go about it.
Setting up a fake door is a 4 step process:
This allows you to understand exactly what your conversion rate would look like without actually setting up the infrastructure to produce and ship a product. Major enterprises do this all the time — you’ve probably clicked something on Facebook before and got hit with this little gem:
After building your Multivariate Test and Fake Door, follow these steps:
Now, you have created your own data.
Warning: you can typically expect a lot of noise in this MVT data (aka, a lot of ads that did nothing or performed just about the same). The goal is to identify the positive outliers — which of them were far above the pack? The more ads included in your MVT, the more likely you are to generate more positive outliers.
Here are the questions your analysis should answer:
With this system, we’ve been able to simulate hundreds of product possibilities for entrepreneurs and tell them exactly which products were worth launching and which ideas to scrap. Happy testing!
