The concept phase is an ideal time for ad testing. But this is not a secret — brands and agencies have been doing it for decades.
The thing is, the traditional methods are outdated. Focus groups don’t offer a realistic environment or user experience and rarely do they actually include ideal target audience members. And A/B testing, while automated and in-market, delivers the shallowest level of data available, telling you only which ad was the higher performer, but never why.
In our new video series, Resting Ad Face, our VP of Performance Marketing, Susan Wenograd, and I share our opinions (we have many) on all things performance creative. And our first episode was dedicated to this very topic — automating creative concept testing the right way.
A creative direction chosen and executed without proper concept testing is the costliest of all. If it doesn’t land with your audience, the time and money spent on concepting, production, and media is wasted.
It’s much smarter to test even some approximation of the idea before shelling out thousands upon thousands of dollars to bring it to life.
The best way to do this is to find a small-scale version of the idea — perhaps a static image ad with the right image and messaging — to prove the concept will drive ad performance.
Having data to back up an ad concept — something that definitively proves your target market will respond because they already have — will send you into production mode with much more confidence.
Imagine this scenario: a new ad campaign is needed for a product launch.
The creative team is given a brief and gets to work on three possible concepts. They come up with a possible ad layout along with a handful of headlines and images that bring each concept to life.
The team brings all three concepts to the client (if this is an agency we’re talking about) or to the leadership team (if this is an in-house team) and pitches the merits of each.
Up until now, this concept presentation has been nothing out of the norm for a brand or creative team.
But what if, at this point, you stated that the objective was not to choose an ad concept — but to approve all three for automated, in-market testing?
This flips the script on the typical pitch process. Suddenly, you’re allowing potential buyers to influence the direction of your ad creative with their clicks and purchases — and not the person in the room with the highest rank.
This is how we move away from making opinion-based creative decisions and toward data-based creative decisions.
Many times, ad concepts are coming to life in parallel to the rest of a campaign — the landing page, the video ads, the social media creative, etc.
Applying learnings from creative concept testing across all of these touchpoints is crucial to creating an impactful user experience and increasing the odds of driving conversion rates. Using the winning headline, images, colors, and more can help create a more cohesive, performant campaign.
Conceptual testing can be used as a new form of real-time market research, feeding creative teams and marketers with data that can be used to elevate the brand as a whole. You don’t have to wait to see how the ads perform to optimize the rest of the campaign. You know upfront which creative elements will drive performance and you can use them wherever your audience will encounter your brand.
Check out the full, 20-minute chat from episode 1 of Resting Ad Face above. And subscribe to our YouTube channel to get the latest episode drop every other Wednesday.
[00:00:00] Susan: Hi, Susan. Hey Jess, how are you? I'm good. Sitting here with my resting ad face on.
[00:00:07] Jess: Yes, same. I've been practicing. I can't do it without smiling though. Today I think we want to talk about conceptual testing. Like you're at the point where you're coming up with ads, you're, you're concepting coming up with ideas for your client or your brand.
[00:00:24] And, you know, there's this point where everything's really, really raw you're creating assets just to, to kind of sell something in. And, you know, I think we want to talk about like why this is like a really good point to test something. So let's, let's just start there. What, are your initial thoughts?
[00:00:42] Susan: So it's, it's timely that we're talking about this because a client I have just launched a new product this week, so kind of the same thing where. It's somewhat related to their initial product, but it's a little bit more of a commitment and it has more selling features, but it's also more expensive.
[00:00:59] So it's like, [00:01:00] it requires more explanation. So the challenge has sort of been what you're talking about, where it's like, well, we could talk about this or we could talk about that, or do we want to sell it this way? Or it's almost like you have too many options of how you could do it. And a lot of the brands I've worked with, they do.
[00:01:16] Really do a lot of testing ahead of time. Like they kind of have to decide this is the direction we're going to take because they have contractors that are creating stuff for them. They're having to set up video shoots. So it's like, you can't just show up and do. Let's just record 18 different things that are completely unrelated to see what works.
[00:01:31] Right. So, you know, I, I feel like sometimes brands really overthink that part. Like you'll have all these, you know, creative strategy meetings and all these briefs and like all of this upfront energy goes into it. And then all the money goes into creating the assets. And then it's the worst is when it's tested.
[00:01:47] And it just like, doesn't move the needle because you just spent all of that time. And I feel like there's this delicate balance that brands are. Kind of starting to figure out, but it's sometimes [00:02:00] difficult to get them there is that like, before you go through all that trouble, can you do a scaled-down version of what it is you're thinking about?
[00:02:06] So like, instead of being like, oh, you know, overthinking everything overanalyzing everything, every little piece of copy, like where the ellipsis goes and like all that stuff that you end up obsessing about later, it's like, what are just the big things. If we had an answer to these big things, it would save us going down a whole path that just wouldn't work.
[00:02:24] Right. So I think those are the things that like brands put so much pressure on themselves to figure out. And one of the things that I love about this client I work with is they're like, just test all of them and tell us what works. So they're the opposite. They're like scrappy do it quick because we'll get data feedback so quick.
[00:02:40] That's like, then they can take that and make something beautiful and perfect out of it. But they're not wasting all of that time and all of that money. And I feel like that. Some brands have gotten there because they've gotten into UGC. So they've kind of embraced the it's going to look kind of crappy, but that's what sells and people want to see reality.
[00:02:57] So there's more of an embracing of that. And there used to be, but I [00:03:00] still feel like there's this pressure of it has to have this thing, or it has to look this way or we have to save these things. And the reality is that your audience is going to decide what those things should be. But if you. Ask them or show them first.
[00:03:11] You'll never know. I think you
[00:03:13] Jess: bring up a good point because like, you may be dead set on like this needs to be video, but you could test that same messaging kind of look and feel and static and get really quick answers back with like low production still in the concept phase and like, no. You can extrapolate that and know like, okay, if this tested well and static, I can, I can pretty well guess that it's going to work well on video as well.
[00:03:35] If, if what you're testing for is like the messaging and
[00:03:38] Susan: the look and feel. I agree. And it helps with things like features too. Right? Cause it's like in static, they're not having to analyze like as a user, they're not having to take in a bunch of input and figure it all out. Like they do with video it's.
[00:03:50] Here's the benefit. Did the person care or not? Right. So even things like that, that can help determine like when the video gets cut together, what should we be talking about? Which we not be talking about. If they seem interested in [00:04:00] this feature. Great show that in the video, don't show this other thing, because sometimes there's there's concepts that are just too abstract.
[00:04:05] It's like, you could explain it in a video, but it's going to take too much work for the user to understand. So it's just, what are the quick wins that are going to get them to the site. So if they want to learn more information, they have the opportunity to, but you're just giving them enough to get them interested.
[00:04:18] And that gets overthought a lot. Yeah. Absolutely. I
[00:04:21] I
[00:04:21] Jess: I know too, like from my agency days, like when we came up with concepts, we were coming up with these assets anyway, like, Hey client, here's three images that work really well with this idea that we could kind of use this style. Here's four or five headlines that give you the feel or the, the message that we're trying to get.
[00:04:38] Right. And typically we're showing a client three different concepts at a time. And so it's like, okay, now we have, you know, four or five different headlines, a few different images. And we have that for like a few different concepts. Like let's kind of set up maybe three different tests and see like, is one of these performing better and that's the direction we'll go.
[00:04:57] Right. Usually, you go into a pitch and a client's like, I [00:05:00] like that concept. And it's like, maybe we can start to flip things or it's like, don't no, don't, let's not pick a concept. Like the objective is today. We're going to show you three. If you're good with all of them, let's test them and let's see what works, right?
[00:05:13] Let's, let's let all three ride and let the customer, the buyer
[00:05:17] Susan: tell us what the, I love that so much better because it's again, it's like, why put that pressure on yourself to be like, this is the one we're going to go with and put all your money there. And that was the test that I launched this morning was a lot like that, where they, they use a lot of UGC because it's just what works for them.
[00:05:33] And so. But it's a mixture. It's like you have ones where the person's in it and photos, whether or not, and photos where there's like a cityscape in the background versus it's just like on an everyday lay flat. So sometimes it's also good to look at what are the assets you have and what are the ways they naturally grouped together that you could test?
[00:05:50] Right. So like, when I looked at the, they sent me a folder, like here's all the stuff we hadn't had a ton, but look for those. Patterns first. So I'm like, okay. So we have [00:06:00] ones where it shows the person in one word, it just shows hands. And then we have ones that are. Urban exterior behind it. And then ones that are, you know, more intimate where it's like on a table or something.
[00:06:10] So if you can kind of naturally categorize those things, you've created so many tests for yourself to begin with. Cause I think that's the other piece is people are like, well, how much is there to test? Really? And it's like, there's a lot if you want to go that way. Right. So that was the, you know, I launched two tests.
[00:06:23] One is with people and ones without and it's, they're kind of up against each other where it's like, here are the people, and then here it is when they are. Showing it with another object, right. So it's just like a lay flat. And then once I see the winner of that, I'll take that and then I'll pit it against like, what if we just show it with an urban background?
[00:06:42] So it's like, there's already three other tests lined up for it. But as we learned this stuff, like they just launched it. They haven't even finished a separate landing page for the product yet. So. As we're learning this stuff in real time, it's like we can feed back the data and say, this headline works.
[00:06:57] This value prop seems to work. So as they're building that [00:07:00] stuff again, they're not just going all in on one concept and realizing later this isn't doing anything for us. Right. And they can
[00:07:06] Jess: build that into the landing page. Right? Like, take a headline that performed the best, take the image that performed the best.
[00:07:10] So that the experience is this really beautiful, like linear thing that, oh, I saw this and I liked this ad and now I'm at this experience that's very, very similar. And all of that was built around insights from
[00:07:23] Susan: your test. Exactly. Yeah. So it's like right now they have just something basic setup to explain the product, but they knew that they wanted to do something a little more in-depth on the Lander and that's part of what we're using the ads to help influence.
[00:07:35] It's like once they get there, did people seem to like seeing a person in it better or not? You know, it's like, were there headlines that like they clicked through because they were interested in this. So we need to make sure that the. page Addresses this, you know, it's, it's those kinds of connecting those dots.
[00:07:48] So it's not just about the ad that's running, but then how do we not only get them there, but then are we delivering on what they saw in the ad to get them through, to purchase?
[00:07:56] Jess: Can I tell you a story? Of course. So let me tell you about how. [00:08:00] When I was in agency world
[00:08:03] Susan: and I love Jess stories. Jess stories are the best.
[00:08:08] Jess: Well. This is how we used to test creative. And I feel like this just illustrates, like how one, how far we've come and two like the level of data that. That is no longer sufficient. And so when I was in my agency days, I'm working on like Kellogg and Kimberly Clark, we would, we would kind of go this conceptual route where we'd bring the client like three concepts.
[00:08:30] Right. You know, here's three concepts for your meatless breakfast, sausage, and you know, which one do you like? And they'd say we like these two, but we want to test them. And so they'd call in You know, some sort of consumer testing agency, we would do some sort of big
[00:08:46] Susan: like focus groups, all of the focus groups,
[00:08:48] Jess: focus groups, nonstop with people who had like nothing better to do on a Tuesday.
[00:08:53] Right. So already not the people that you probably are looking to, to put this in front of. [00:09:00] Two We used to print out the concept. So like, w I'm dating myself now, but like they would print them. And I remember it was really exciting cause there was a point where they would actually be able to see them on a computer, like, you know, after like a year or so it was like, oh my gosh, they're actually going to get the, see them like on screen.
[00:09:19] And, and so they, but they wouldn't actually like put them in an environment or anything like that. It was like, here's the ad What do you think? Right. So again, like now. You're each time you remove a step of like the reality of it. The data is just completely skewed or, or totally invalid. So I think like what you end up getting is like people said, they liked this. And then the focus group company would be like, here are the optimizations you need to make based on what people said. Right. And so now we're optimizing creative. That was poorly tested, right. It Spirals out of control. Yeah.
[00:09:58] And so like, I think [00:10:00] now, like that's like an unacceptable level of, especially now when we have, you know, iOS 14 privacy concerns, like all of these things that are blocking us from seeing our, our target audience now Like you have to know what they want to see. You can't leave it up to chance or, you know, some kind of extrapolated version of traditional testing.
[00:10:21] Like it's just not going to cut it anymore.
[00:10:23] Susan: I think that that's also another point that I think has made things hard is because we can't be as targeted, as busy as we used to be. It's like, we used to be able to make, you know, all these different versions of these ads to appeal to the group that we were targeting right now.
[00:10:37] it's like I mean, you can still do that, but Facebook just struggles to optimize that way. So you're forced to go broader. And I feel like that's forced a lot of brands to have to start from scratch a little bit on there. Cause like they knew what used to work for these little niche audiences. But once those stopped performing well on Facebook, because they just weren't large enough for Facebook to get enough data.
[00:10:59] [00:11:00] They're being forced to go larger, but then it's, it becomes a question of how do we make stuff that has more mass appeal, where we're not relying on aligning like a really specific message to a really specific. You know, persona type, we just cause you can't do it anymore. So that's been, the other interesting thing is it's like, and this brand definitely went through that where it's like, because they had so much UGC.
[00:11:19] I mean, we could set up really specific groups and the two would marry together perfectly, but it has not done that. Well, it's like maybe some interests work, but they're so broad. I mean, there's. Such broad interests that it's like, you could still probably so show general creative. And I don't know that it would perform any better than the stuff that's like hyper-specific to that interest.
[00:11:40] You know what I mean? It's more so for like budgetary purposes, but I think that's been the other challenge with the conceptual testing is that. In some ways there, they wind up being able to do less versions than they did that are more general, if that makes sense. So it's like, instead of having like this one concept, but like we changed the headline to six different ones to [00:12:00] align with these targets.
[00:12:01] Now it's like, we have six different versions, but they all have to speak more generally. So it's a different kind of testing in that way because the audiences just don't have the impact that they used to. So I feel like that's kind of, the other piece of it is it's like, how do we make marketing that makes more people happy because that goes against What we did forever. It was like, no, I want moms that are dog walkers that, you know, watch reality TV. And it's like, this is what we're going to show them. And now that doesn't work. So it's like, okay. So how do we just make something that women will like? Cause we don't know which women are going to be looking at it.
[00:12:32] Right. So we
[00:12:34] Jess: Or how do we add little nods to all of those things. We know they like in our creative because we can't reach them with targeting, but we can reach them with creative. Right.
[00:12:43] Susan: Have we put little, like, how do we signal? We know you. That's always interesting. Cause it's, you know, with the targeting too, it's like the optimizations affect all that.
[00:12:53] So it's like if you have stuff set up and you're telling Facebook, like find people that will convert to your point, it's like, we're [00:13:00] pretty sure we know who those are, but it's definitely going to people that aren't those people which is fine. Cause it sells. But it's like, we just, I feel like we know less about our users than we used to.
[00:13:10] I mean, not necessarily who. The market is, but we used to have a lot more detail. You don't like, I remember back in the day where it's like, you could take your email list and upload it to Facebook and it would give you a breakdown of like everything. They like, what they do, where are they located? I mean, it was like, it gave you all of this information and of course they got rid of it.
[00:13:31] But so you used to be able to learn a lot. You know, if you had some of your own data, you still able to learn a lot more. Like if you just had an email list, you could learn a lot about like, oh, they watched these shows or, oh, this is what they do in their spare time. Or this, these are their age ranges.
[00:13:44] So it's like you used to, there was like a two-way street of how I used to be able to learn that stuff. And it's just not there anymore. So I think that's, I feel like the testing piece is still one of those things we have that gives us that feedback. It's just interesting to test things now because it's, you're kind of [00:14:00] like, there's like six more general concepts and we'll just see who likes what
[00:14:03] Jess: yeah, exactly.
[00:14:05] I think. Creatives, you know, like more coming up with these concepts, we have like our heart set on one. Right.
[00:14:12] Susan: We love one it's out of there.
[00:14:14] Very emotional
[00:14:14] Jess: people. Oh yeah. And so it's like, this is the one I want to win. Right. And, and this is the one that I'm really going to sell hard. And then I usually come up with a safe one because I know like that's what the client wants to see. And then. Crazy one out here. Right? And so you have like this, this spectrum, and I think what, like the, this new kind of way of testing, or like the fact that we have automated creative testing available to us now is like one, you can.
[00:14:41] You can still have those feelings. Obviously, I don't think any creative is ever going to let go of those feelings. You can validate them or, or, you know, find out that maybe you didn't know and you still learn something that way. And to like, it feels good to be able to like go to a client and be able to say like, Hey, this is the one that we would [00:15:00] recommend typically, but like, let's really see what your customers want.
[00:15:04] And I think that just ups the level of credibility, whether you're an in-house team or an agency, I just love it up to your credibility of like you care about the performance more than like the preciousness of your
[00:15:17] Susan: idea. Yeah, I totally agree. I've told you before about, you know, I, I used to be more on the creative side where I was a writer.
[00:15:24] And like, that was the part that drove me. Craziest was I was like, we should just be doing whatever works. Right. But it was like, you get very involved with creative directors that are like, no, I want to use this word instead. And did, and I was just like, why don't like, why are we arguing about this? Like, why?
[00:15:40] Like, it just felt like such a huge waste of time and energy. I'm like, why are we arguing about. A fictitious ad run yet about this. Right. And that was why it was funny because that was what made me end up like the marketing side better. Like I was doing copywriting for email marketing and I was like, you can test it.
[00:15:59] [00:16:00] Like I thought it was the most amazing thing ever. I was like, you get split test things. It was like, my dream come true. So it's the same, like, I feel like this fixes a lot of that. Even before, like, you know, even you would do creative testing before it was still flawed. Like that's the nice part with our pipe is that it's like, you'd still get the people that are like, well, I think it just worked well because that's the image I said would work.
[00:16:22] And it's like, you didn't know if it's the copy you battled for that work or who is the, like, you really couldn't ever settle that argument or have an agreement about. This is the words that work or like these are the visuals that work because it was something else. Yes. It was like always inherited something else.
[00:16:39] And you're like, what if it was that one thing? Like, what if we run this again and it bombs, right. So I think that's been the other piece, like to your point, you get performance data with. And just as an efficiency person, I feel like places that adopt that they, they stop wasting so much time arguing about this stuff.
[00:16:59] I mean, it's like [00:17:00] hour-long meetings arguing over a headline. I mean, you've been there and you're just like, oh my God, like we could solve this in a day. If we just throw a thousand bucks at this and run it, like we could settle this argument in a day, you know,
[00:17:11] Jess: Case in point, talked to a Marpipe customer yesterday. Yeah. And they brought up how, like they have been using an all-white version of their logo forever and they decided in their test. I think actually our customer service team success team was like, why don't we test your full color and your all-white version against each other? And they dropped CPA by $60 with the full-color logo. And like, they calculated them. It's going to save them $240,000 this year, based on like the number of, you know, new, new customers are trying to bring in. So it's like, not only do you save this time and these arguments that are like so arbitrary, but like you're literally saving potentially hundreds of thousands of dollars to not argue about
[00:17:58] Susan: this anymore.
[00:17:58] Well, I mean, and think [00:18:00] about like, what was the there's like a calculator online that it's like, how much is this meeting costing us, where it's like, you can plug in everybody's salaries. It's really essentially that too. It's like, do you want to pay the money for all those people to sit in a room and argue with no data?
[00:18:14] Or do you want to just take that same money and stick it to test where they'll get actual data that will tell you what to do. And then. There's none of that friction and contention that you get between like the creatives and product managers. And like, there's always that push and pull between the groups.
[00:18:27] And it's like, it just kind of settles that and removes it as a thing. It just becomes this emotionless thing where it's like, we put it in the machine, the machine will give us numbers and that's, what'll determine what we do going. Absolutely.
[00:18:37] Jess: Yeah. That calculator was a deep cut on the internet. That was, that was a really good referenceI forgot about.
[00:18:42] Susan: Thank you. I only spend like 10 hours of my day and you're set. So, you know, I've got that. And then I've got the, you know, the website where all it is is the Nova. Where you just like another deep cut, but like this is necessary. Internet [00:19:00] culture, my friends,
[00:19:01] Jess: you have to know these things. Have you seen with any other clients or this one we've been talking about in particular that they're testing concepts against different strategies. So like for prospecting or for current customers or, or anything like that and we're playing out.
[00:19:21] Susan: Yeah, it's a good question. That's actually been on the media side, like one of the big, huge debates since iOS 14 hit is that you can, I mean, you obviously can still run remarketing.
[00:19:33] The audience is so much smaller than it was because when people, once people opt out of being tracked Facebook doesn't know, like you can't treat them as a remarketing audience, they'll just get lumped into top of funnel. So it's like if they've been to your site or whatever, and you're like, oh, I want to re target field live into my site.
[00:19:49] It's not going to be everybody that's been in their site. Like it's going to be people going to top of funnel. So a lot of us still run remarketing as like a best practice. But many of us [00:20:00] have stopped excluding those people from top of funnel, just because we're like the less exclusions, the better it seems to do.
[00:20:07] It's like, as soon as you start adding exclusions, it just seems like Facebook starts freaking out and it's like, it just doesn't, it doesn't do it. It's like the more people you can have in a pool, the better. Yeah. Like a lot of times what people will do also is sort of the same thing in remarketing where it's like, they're trying to make that pool as big as possible.
[00:20:22] So, you know, back in the day, we might've had remarketing for people that visited the website or remarketing for people in our email list. A lot of times we're making those audiences. And then we're when we make our ad group targeting, we're just putting all of them in. So it's like, we're not treating site different because it's like, you have to just make it as big as you can a to make sure it serves.
[00:20:41] But B that it's big enough that Facebook is getting some kind of data feedback loop from. So a lot of us it's like, we still do it, but it's just, again, it's more general than it was. It's like, we're not, I mean, I remember we used to have like people that visited our website the last 30 days, the last 14 days, the last seven days, and you'd exclude them all from each other.
[00:20:59] Like, it was a [00:21:00] whole thing. I mean, remarketing was a whole separate setup and now it's kind of like, it's basically anybody that's visited your site in the past seven days. Cause it won't track past that. And then like anyone you, you add from your email list, you know? And then like the only time I, I do see it different sometimes is.
[00:21:16] If you do remarketing, based on the fact, they just like, just engage with your ad or watched a video. Like, since they're a little higher funnel, sometimes, you know, we'll still kind of house them separately usually, but that's even changed. So, you know, we'll still test remarketing creative. It's just hard because there are a lot of people that have been to the site that are getting top-of-funnel creative now.
[00:21:36] So that's also been a little broken.
[00:21:38] Jess:
[00:21:38] Exactly. Well, that was fun. Let's do this again.
[00:21:42] Susan: We'll totally do it again. We'll come up with another topic and we will rant and rave. Awesome.
[00:21:47] Jess: See you next time. Resting Ad Face, everyone. See you next time. Bye. Bye.
