It’s the number one question we get from marketers:
“How much should I spend on ad testing?”
The answer, as with so many things in marketing, is “It depends.” There are plenty of factors to consider, including:
It’s a very chicken-and-egg situation — with many chickens and eggs involved. This is why, in episode 6 of Resting Ad Face, our VP of Performance Marketing, Susan Wenograd, and I talk all things budgeting when it comes to testing your ad creative.
Good news: if you have a media budget, you already have a testing budget. A solid guideline is to use somewhere between 10–20% of that budget for ad testing.
This will allow you to pinpoint your top-performing ads and creative assets. Then, by scaling the top performers from your ad tests, your remaining 80–90% will work that much harder for you.
You can calculate the potential lift in conversions for your brand with our ROAS calculator.
There’s often a perception that carving out a testing budget is a sunk cost. That you’re collecting data while sitting idle, waiting to push ads live while you collect that data.
But with today’s ad testing tools, your tests run live directly in front of your actual audiences. This means any conversions, clicks, or leads generated in your tests are also driving your bottom line.
Your audience is simply “voting” with their clicks and purchases to show you which ads and creative elements resonate with them the most.
Your test results can also help you get more bang for your media budget in less-trackable channels.
For example, perhaps you use an image that drove clicks in your ad test on a billboard. Or maybe you use a piece of copy that decreased CPA in your ad test in an audio spot.
By using top-performing creative elements in environments that aren’t as easy to measure, you can launch with greater confidence.
For more, subscribe to Resting Ad Face on YouTube or your favorite podcast platform.
[00:00:05] Jess: Hey, everyone. Welcome to Resting Ad Face. I'm Jess Cook, Head of Content at Marpipe and I'm here with my amazing co-host Susan Wenograd.
[00:00:15] Susan: Hey, everybody. I am the Performance Marketer in residence at Marpipe.
[00:00:22] Jess: That's right. And we are here to rant and rave about all things ad creative. We're getting really serious. I mean, check this out.
[00:00:31] Susan: She's got a real microphone and everything. I swear mine will hopefully be ready by next episode.
[00:00:36] Jess: Things are happening. Real, like, actual polished production things. Which is exciting. We're becoming official.
[00:00:46] Susan: We are.
[00:00:48] Jess: And the other exciting thing we have been named one of the Top 50 Marketing Podcasts by Guru Events, which is
[00:00:59] Susan: Mind blowing to me. [00:01:00]
[00:01:00] Jess: Mind blowing. So thank you to everyone who nominated us for that. Cause that's crazy.
[00:01:07] Susan: That was a nice little surprise. It's like, you always hear about people like, "I've been podcasting for two years. It's done nothing." And Jess was like, "Hey, we got into this top 50 list," and we're like, What?
[00:01:18] Jess: What?
[00:01:19] Susan: What? Okay. Not gonna turn that down.
[00:01:22] Jess: So they're doing like a bracket thing, kinda like a March Madness type deal. So if you feel so inclined please look for Guru Events on LinkedIn. And whenever we pop up in that bracket give us a little vote. That would be amazing. Just to get some, some more ear, I almost said earballs. Eyeballs.
[00:01:44] Susan: Well, it's kinda both because you can watch and listen.
[00:01:46] Jess: Earballs yeah. More people to see and listen, and use their earballs to hear and see us. I'm fresh off of vacation. It's obvious. So we are here [00:02:00] today to talk about budgeting for creative testing. I think something that, you know, our customers or new prospects come to us a lot and they always wonder like, how much should I be putting toward this.
[00:02:12] Susan: God, this is like probably the number one question we hear. I feel like people ask that before they even ask what it costs to use Marpipe as a tool. It's such a gray area of like how does this fit into what I currently do? You know? And I, I, yeah, especially as media buyers, it's a lot of times we're given this is your budget for the month.
[00:02:31] And so we just think about it as this pot of money. So to your point, when we bring up budgeting and there's this dawning realization that, okay, these are gonna be a separate campaign effort, I'll have to carve off money for them. Then it starts to get like, whoa, how do I do that and still hit goals? It just brings up a lot of questions.
[00:02:49] Jess: And, and rightfully so. I think it's kind of a blocker for people even to want to try testing sometimes because, or a new form of testing that maybe they're not familiar with because, [00:03:00] you know, they, they're thinking they're gonna have to dig up this huge pile of money in order to do it.
[00:03:05] And, and really realistically you can get results with you know, smaller budgets than I think people realize. But to start, I think the really good news is that you don't actually have to go out and, and find more budget for this. I think, you know, something we recommend to people is you take your existing budget and you carve out 10 to 20%. Depending on what your budget is and how many variants you wanna test and, and all of those variables that play into it.
[00:03:34] But I think the good news is you already have the budget. It, it's part of your media budget. And you just need to kind of make sure that you're, you are prioritizing testing and carving some of that out for testing. And then what happens is the kind of 90 or 80% that's left over once you've done that testing is going to work so much harder for you because you were able to find some elements or some ad variants that worked harder than what you would've put out, had you not tested. [00:04:00] And so now that that, you know, left over majority of your media budget is now going to perform that much harder for you.
[00:04:09] Susan: I think there's also this perception too, that because I'm putting it into a test budget, there's this assumption that that means it's not gonna perform, you know, like I, right. I feel like as soon, and I say this as a media buyer, trust me, like, as soon as I'm like that as a test budget, it's like, that will not be driving performance.
[00:04:29] Like there's something in my mind makes it feel like it's this other thing, but that's usually not the case. Right? I mean, sure. You may run a test where it's the majority of things don't work, but I have just not found that to be the case. If, if you're a brand knows enough about your audience to get started,
[00:04:47] you sort of know what to start with and then what to build on. So it's very rare that I've run a test where it's like, it just fails outta the gate and it's doing horribly and, and in those cases it's like, okay, then stop. [00:05:00] There's nothing that says, like you have to spend the full seven or 14 day if you're, you know, halfway through and it's just, no version is doing well, you know, all the, the
[00:05:10] versions of every variant are just neutral or low, nobody says you have to finish the entire thing. But it's so rare that I, I run into that. So I think that there is this tendency to think, okay, this is a test budget and that's all it's good for, but you still get sales and performance off of it. Yeah.
[00:05:28] So I feel like I always have to kind of remind customers of that cuz to them, and I, I think it's a good sentiment, actually, it's if you're gonna test with money, be okay not getting a, a return on it. You know, it's like, that's worst case scenario though, and it's not the usual. But it's like, you do kinda have to go into it
[00:05:44] with that mindset of like, okay, how much money would I be okay just losing? But know that that's not the reality, just because it's a test budget doesn't mean that it's not driving to your bottom line.
[00:05:53] Jess: Absolutely. Here's the thing, those conversions, those clicks, those impressions all still count just because we're [00:06:00] using test budget.
[00:06:00] Doesn't mean they're like not real. Yeah. I think that's the really nice thing about like Marpipe and multivariate testing is you're putting it in front of your actual audience. It's not some panel of people or some group that doesn't really pertain to you. It is live in market. And, you know, again, tho those, those conversions and those clicks, they, they count, they, they go towards your performance and, and that is media budget well spent.
[00:06:27] A little tool that we created that can kind of help show that and I'll drop a link to it in the, the show description, show notes is a ROAS calculator.
[00:06:38] And it, it's kind of cool. You get to see like, okay, if I took this percentage of my budget, you know, we recommend 10 to 20% and I, you know, ran this kind of test and I get to actually see like, oh, here's my CPA. If I don't test here's the total conversions I get. And here are the total conversions I get, if I do.
[00:06:57] And that that's proof. Like, all of [00:07:00] that testing is still going toward the bottom line when you are finding, you know, those, those top performers.
[00:07:06] Susan: It's also helpful too, just because as media buyers, we deal in such concrete numbers. That a lot of times it's the question of like, It's not that anybody disputes that you should be testing stuff, but it is that fear of like, is this gonna negatively impact?
[00:07:20] Like, I need to be able to do this while still delivering on goal. And also just kind of like, what can I expect from it? And it's very hard for us to answer that without knowing the math. And that was actually why the calculator became a thing. Everyone asks this, can we just give them something that they can plug it in? And that way it can be tailored to their situation, right? I feel like that was always the gray area and always the question that was very hard to answer because it is so individualized. It's a handy little tool, though. You guys did a great job making that.
[00:07:50] Jess: Thanks, Susan.
[00:07:53] Susan: You're welcome.
[00:07:56] There's also the ultra scientific approach to testing. And then [00:08:00] there's just the reality of being a media buyer with a brand or, you know, clients that have performance goals. So a lot of times people go into it thinking, okay, I have to run this test and it has to run for the seven days or 14 days that I picked it for.
[00:08:15] And I just gotta like buckle up and just let it run. The thing is, when you've worked at the brand long enough, you, you kind of know where those baselines are. So a few days into launch, you're probably gonna start seeing like, hey, these things are keeping up with what we normally see or they're not.
[00:08:33] I work with a brand that about 70 to $80 CPA is what we expect. So if I'm running a bunch of ad sets in Marpipe and there are some that are firing it off that, at that rate or better. And then there are some that are like four to $500 CPAs. Those four to $500 ones are never gonna catch up. Like I just, I know they won't, cuz I've worked with the brand long enough that it's like, it doesn't get better.
[00:08:56] It's like it either achieves it in the first three to five days [00:09:00] or it just it's a failure. So much to the chagrin of the highly scientific engineers in the world that are like, "No, you have to let it ride. They all have to reach the same amount." I get it. But I'm also kinda like, I know this brand well enough that it's like, I, I can tell when something's working.
[00:09:16] I will go ahead and just turn off those versions that are not producing and will never catch up. And a lot of times they do have something in common. So it's like, if I look at those versions and see what's in them, a lot of times it corresponds to the stuff that's not doing well. And if there is a part of it, that's doing well, then I'm like, okay, maybe I need to flag that, like the image was fine, but this headline on it and this paragraph text is not doing it, right? Lesson learned, but I don't need to keep running everything else that's failing.
[00:09:48] I'd rather just concentrate the budget on the stuff that's working. Cause the other thing is too, it's like at that point you're taking more and more stuff out of rotation. So it's like you're saving that budget. You are concentrating on the [00:10:00] stuff that's working and then it's like, then you can take the, you know, once you reach that time period, you can either say, okay, I've reached seven days.
[00:10:05] I wanna turn it off. Or I wanna just let it run. Or once this is over, I just wanna scale it cuz everything that's left in there is a winner. Right? So the test campaigns that are created and the ad sets, I don't treat them as like this thing that just exists and it has to be. To me it's still a living, breathing campaign in a lot of ways.
[00:10:20] I give it more grace than I would as far as like how long it runs, because I know that it's serving a dual purpose. But I don't fall into the camp of like, you must run it no matter what, because at a certain point you've probably learned the lesson without spending anymore money. Like you just have to use your, your best judgment with that. It's not a one size fits all.
[00:10:38] Jess: Yeah. And I think it's, you know, yes, this is the scientific method applied to ad creative, but there's also this aspect that there's someone responsible for that budget, right? It's not a straight experiment where we have to wait and let it ride and see where all of the chips fall.
[00:10:57] Like you say, if you know your brand well enough and you know that [00:11:00] like something's not performing. Turn it off, give everything else a little bit better chance of, you know, that might be performing of getting a little bit more spend. Right. So I think that, that makes perfect sense. Yeah. Susan, you have some really interesting thinking around like how to use creative testing as a barometer to spend the rest of your media budget.
[00:11:23] Can you share some of that?
[00:11:24] Susan: Yeah. So I think this has also gotten interesting with the way things have changed with iOS 14 on Facebook. Because a lot of times on Facebook you'll hear from media buyers and it's certainly been the case that I've noticed
[00:11:37] is consolidated is just better. Like, same budget, less ad sets, probably less campaign types, less granular targeting. And so in some ways it makes budgeting easier because you're not looking at it like, okay, I have to spend this much. And I have it, you know, spread across 50 different campaigns and ad sets because doing it that granular just doesn't work anymore.
[00:11:59] Right? [00:12:00] So you might have the same budget, but now it's across like five to 10 or whatever it looks like for you. So the thing that, you know, that I keep in mind with the creative testing is it's what can I learn that will give me the best insight from this consolidated world? Right? I mean, if, if I had used Marpipe back before iOS 14, I'd probably be running the same amount of budget for tests, but I would be running more tests at a time. The thing that I think now we do differently than we did before was because we had all that audience targeting, we'd kind of know, like alright, we can scale this audience to this amount.
[00:12:40] And then we start seeing, you know, efficiency erode et cetera, et cetera. Now, because we have such broad audiences, the audiences don't erode as quickly with performance. Like they don't get burned out as much, cuz they're so large. It's made creative testing a little harder because you're having to make creative for a [00:13:00] whole bunch of different people.
[00:13:01] Whereas like before you kind of knew, like, these are people that like this, or these are parents, or these are college students or whatever. If it's too granular of a group now it's just kind like broad targeting maybe by gender and age, sometimes interest still work. I mean, it has, it's certain situations.
[00:13:17] You're definitely speaking to a larger group. So figuring out how to find a message, that's gonna be interesting in all those people is really hard. So in that way, I feel like there's more creative testing in regards to versioning. If that makes sense, as opposed to like more creative testing where it's less versions to more audiences.
[00:13:38] So it's kind of like, maybe you had like 20 targets and you were testing two to three creatives in each. Now we have these great big broad groups and you need to test like 10 to 20 things to figure out what resonates for such a large group of people. So it's, it's changed in that way. And that's also, I think what has made budgeting very interesting because before, like [00:14:00] I said, you kind of know, oh, we can spend X amount per day.
[00:14:01] When we go above that, it kind of burns out. And now I find that it's not as predictable. Like there's definitely with the, with these larger groups. I also noticed there's there tends to be trends that are more universal. Everyone's kind of buying or everyone's kind of not buying and there's not anything in between.
[00:14:22] So trying to manage your budget to those trends along with finding the creative, it just feels like the creative testing has become so much more important because you're trying to reach these large groups of people, you're trying to capitalize on when they are buying as much as you can. And then, you know, the other thing that everyone has encountered is it just takes longer to get data on it because Facebook's getting like 30 or 40% of the data sample it used to.
[00:14:49] So it takes like that much longer to get large enough audiences for it to even have a clue of what's working and what's not. Yeah. So it's made budgeting interesting because it's just not as [00:15:00] cut and dried as it used to feel like to me. And it, it changes quickly and it takes longer to learn. So the creative testing piece for me, I feel like shortens the gap on a lot of that.
[00:15:11] Jess: Something you've shared with me before, too, is this idea that like, it's a, it's a cycle, right?
[00:15:15] You, you use, you know, some of your media budget to test and see what's working, but then you need to use those test results to dictate how you spend the rest of the budget, right? Yeah. Or, you know, once your, your ads are running, you're scaling and you're like, oh, this variant that performed well in testing is doing, especially well here on Instagram or LinkedIn.
[00:15:35] Talk to me a little bit about like that cycle of how you then use those insights and like what you see is working well to have a more fluid approach to media budgeting.
[00:15:48] Susan: So we might test something that we're like, wow, this did really well. We're, we're scaling it. It's continuing to do well. Those learnings translate to other things. And those can drive other budget decisions of things [00:16:00] like, hey, you know, we're not getting this creative to get traction in LinkedIn or Google or whatever it might be.
[00:16:06] Why don't we move the budget over here because it is working well? Right? So it's like, you can kind of move those things around like chess pieces. A lot of times it's a little more complicated if you're you know, in an agency setup. Because sometimes you don't manage all of the paid media. You know, I certainly have clients where it's like, I just do their Google or I just do their Facebook and they have someone in house or whatever the setup is, someone else is managing those channels.
[00:16:27] So it's a little harder in those instances. Cause you can't be like, hey, I wanna gank some of your budget from your other channel. Cause that person's like, no. But if you're overseeing all of it and it's kind of like a holistic thing, it's helpful. And that's been the other piece that I think has been interesting is that so many, especially in eCommerce, brands, they've moved from this channel's driving this ROAS. Now
[00:16:50] it's media efficiency rate is what everybody looks at, which is basically like, here's the amount of marketing dollars we're putting in. And overall here's what we're getting out of it. And since [00:17:00] it was moved to that, it there's a lot more open-mindedness about moving around the money a little bit to figure out like, what is that right mix?
[00:17:06] Like if we double what we're spending on Facebook, do we start to see that MER drop? If we, you know, if we increase it by 30%, do we see incremental gains? Right? So, that can also help if you start finding those things that are doing really well. But we also have seen it where it's like something we, you know, we tested Marpipe, something worked really well and they, you know, they redid their site to include that language. Or we have a client like their billboards are gonna use the headline that they, that just killed it for them on Facebook. So it can also really fuel your other budgeting efforts when you look at it that way, because you're like now we have something we feel confident about spending money on even in environments that might not be as trackable.
[00:17:42] Jess: I think it used to be that like somebody maybe on the finance team would work with like the media team to like, this is your budget. Right? And then it's like, okay, we're gonna allocate this toward digital and this toward traditional and whatever. And it wouldn't deviate from that.
[00:17:57] No matter the performance, like that was just the [00:18:00] budget. Yeah. And now I think we have so much data now at our fingertips that it's like, it doesn't have to be that way anymore. We can move things around based on testing, based on performance. And we should. Like, it's irresponsible not to.
[00:18:12] Susan: Well, and at so many brands too, like they have options to look at the stuff in ways they didn't before.
[00:18:16] I mean, there was really, all you had was, you know, you had to UTM tag everything and you looked at it in Google analytics. And then even then it was like, they didn't even have attribution for a while. It was like, it was just last click. So it's like, if something was driving really great awareness and they were converting because they searched for the brand and they came to the site, it always looked like it wasn't doing anything and people would wanna cut it.
[00:18:36] And there were a lot of times. You know, I had brands where they're like, well, it looks like Facebook isn't doing anything. We're just gonna pause. Like, I still remember there was this one brand, like it's not doing anything, the CPA's too high. So I'm like, okay, great. Let's, let's pause. I mean, I'll test anything, right?
[00:18:49] Yeah. So we paused it within three days. They're like, we have no brand search, please turn it back on. So it's like, they learned kind of what it was and wasn't influencing, you know?
[00:18:57] Jess: Yeah, I think that's such a great [00:19:00] point around like, you turn the ads off and then all of a sudden there was no like brand search. Right? And, and how would you know that? And I think, I think the really interesting thing to consider too is like dark, social mm-hmm people see your ads, they don't click, they don't buy, but now they know about you.
[00:19:16] Susan: Yeah.
[00:19:16] Jess: And the next time they see it, right. They're going to,
[00:19:18] Susan: Eeah, we see that with our stuff. I mean, we see that even with Marpipe, it's. We, you know, we had our, this was like the really fun few days, but we randomly had our Facebook ads account disabled, which like every brand I think has gone through at this point, why isn't this?
[00:19:32] I always see all these panicked people that are like, we didn't do anything. And I'm like, it is just appeal to come back on. So we got it all fixed, but it was like, there was like a four or five day period where it was down. And it's so funny. Cuz if you look at that week, our Google stuff is down. Like our brand defense on Google, even that like we saw the clicks go down.
[00:19:50] So there's definitely a halo effect there.
[00:19:53] Jess: It's the opposite of rising tides lift all boats. It's like sinking tides sink [00:20:00] all the rest of your marketing ships.
[00:20:02] Susan: Exactly. It's like, it's gone. Therefore, nothing else exists. And it's hard. Cause everyone knows that they need to have brand awareness.
[00:20:09] I still think that's the hardest part for a lot of brands is they're. These are the things that I know are doing well for ads for direct response. I think the part that will always be the art form is the brand awareness piece. There's just a part of that. That is like an art form. It's like the, just do it of Nike's it's the, yeah.
[00:20:25] You know, it's like, it's all that the, the timeless commercials by MasterCard, right? It's like, those are the artistic things that you'll never necessarily be able to. You know, measure, but it's also like, that's also the stuff that doesn't do well on the channels that you , you tend to put like direct response media into, right?
[00:20:42] Jess: Yeah, the goals for that ad creative versus like performance or conversion, creative, like they have to be so different. And, and the, the rationale for the budgeting there is totally different too, for that reason. Right? Because one is just overall awareness and trying to get people to know you [00:21:00] so that someday, like they want to buy.
[00:21:03] And, and the other side is purely. We just need people to buy or sign up or whatever, whatever the goal metric is. Right. Testing for both of those is, is still crucial, right? Yeah. But I think the goals for them have to be
[00:21:17] really clearly decided upon before and, and, and buy in on that is, is so critical because it's really hard to get people to buy into the fact that like, we're just gonna run these ads and expect really no action on them.
[00:21:34] Susan: Expect nothing to happen.
[00:21:35] Jess: Right. Yeah. Except the,
[00:21:36] Susan: They'll be a great success but nothing's gonna happen.
[00:21:38] Jess: Right. Hopefully something in someone's mind changes. Right? And that's hard to buy into.
[00:21:44] Susan: Well, I think that was everything we wanted to cover today on the topic of budgeting. Who doesn't like talking about money? Thank you, everybody that listens. Please feel free to give us a review and a subscribe to help us out if you are so inclined. Jess is gonna link to that ROAS [00:22:00] calculator in the show notes, if you wanna see it. And we also have some blog posts about some of the stuff we talked about today, too. So feel free to go
[00:22:06] Jess: I'll drop those in there.
[00:22:07] Susan: What was that?
[00:22:08] Jess: I'll drop those in there.
[00:22:09] Susan: Yeah. Exactly, good call. She'll have those in there. You can also join our newsletter if you wanna get those blog posts updated to you. So, you know, when they're coming out and kind of what's happening and recaps of these episodes. Feel free to sign up at Marpipe dot com. And that's all for this week.
[00:22:24] We will see you next time.
[00:22:26] Jess: Bye
[00:22:27] Susan: Bye.
