Howdy Haveaclues!

I hope you’re had a fun and/or relaxing weekend! Take a minute, take a breath, take a sip of a tasty beverage that is appropriate for the time of day you’re reading this.

I recently did an interview on the Marketing Mindset podcast and it's now available on YouTube and wherever you get your podcasts. I had a blast recording it and I think you’ll enjoy it. We dove into the practical aspects of implementing "Ugly Ads," the proliferation of post-it ads on DTC Twitter, my campaign testing structure, my ideal mindset for media buyers, and my thoughts on what marketing roles will have the most demand or opportunity in the coming years, annnd so much more. Let me know if you watch or listen!

In last week’s newsletter, I offered up some free stuff for my subscribers and I wound up giving away thousands of dollars worth of merch, my audit template, CreativeOS templates, and consulting calls to some of y’all. Stick around until the deep end of this email, but don’t skip ahead, I miiight have something to give away!

Today, we’re going to launch into the first of my bajillion-part series:

Last week, I lightly touched on the “breakdown effect” and a bunch of you asked me for me to dig into it more. I also put this poll on Twitter:

62.9% of people don’t know what it is or want to know more about it. Is that number lying? No. Is the qualitative side of this data questionable? Maybe 🤷‍♂️

Today we dig into how seemingly clear performance data can be misleading due to black box machine learning systems.

Before we go further and talk about the breakdown effect, is today’s topic even for you? We’re about to dig into the weeds on ad data analysis. If you never need to analyze ad data or talk with anyone that ever does, then you can skip most of this email. If you’re subscribed to this newsletter and you have no interest/need for this topic and you’re not from my mom (hi mom!) please smash the reply button and let me know this isn’t for you and tell me what you want more of from me.

The breakdown effect comes into play any time you let the system optimize your budget to get your more of your desired actions. I generally recommend letting Meta’s system optimize because it can do so better than your feeble human brain can do manually. Examples of areas the system optimizes budget for you:

When you look at individual ads in an ad set or breakdowns, you think you’re looking at raw performance data. It’s easy to think “I spent X and I got Y back.” This is technically true, but when the system is optimizing budget between ads you shouldn’t compare that data to another ad’s, they’re not a level playing field. It’s not apples to apples.

It’s nuanced.

Speaking of nuance, if you enjoy the complex nuances of media buying, ad data, creative, CRO, and more, you’re gonna love this newsletter’s sponsor: Foxwell Founders.

It’s the perfect community for business and agency owners/leaders, freelancers, consultants, media buyers, creatives, and CRO specialists who want a private place to talk about any and all of these topics with others and experts.

There’s no spam, no selling, and no anonymous trolls (unless you count me as a troll, sorry!).

There is tons of information/experience sharing, intimate cohorts, group calls, supportive feedback, useful resources, and community events.

I owe Andrew and the Foxwell Founders community a huge thank you for helping me make some amazing friendships and for helping me launch my account audit template.

If this sounds like it might valuable for you and you want to join me and the rest of the incredible members, click here to get started. You can always try it for a month or two to see how it feels. I’m sure you’ll get a ton of value out of it!

While I don’t have a discount code, if you’re looking for a deal on a Founders membership, Andrew told me to have you email him directly: [email protected] and because you're on my email list he'll hook you up.

Back to the breakdown effect!

Looking at this screenshot of ads sorted by ROAS, one could easily say that top three ads are the “best ads” because they have the highest ROAS and lowest Cost Per Purchase, but in reality, the “best ads” are the ones that were able to scale the most in green.

Each ad is going to be relevant to different users in different ways on different platforms and placements at different times, but you can only see the end results, not the reason why/how it was optimized that way.

Some ads might do really well with a smaller sector of users, while other ad might be able to perform with waaaaay more users. Sometimes those users overlap, sometimes they don’t.

For example:

They potentially overlap some of the same users and Ad A would be able to achieve a much higher ROAS at a lower spend, but Ad B might be able to more efficiently distribute much more of your budget.

In reality, the differences are actually much more complex than the simple attributes like age, gender, and interests in this example.

While we humans can try to interpret creative relevance to certain demographics, we can’t know how much ad inventory is available, how competitive it is, or how it compares to other potential demographics or dimensions.

Now imagine how many times per second Meta’s system has to figure out the right ads to deliver to the right users for the lowest cost.

You shouldn’t compare the ROAS or cost per conversion of ads that received different amounts of spend. You should (mostly 😅) trust that the system is trying to get the lowest possible for your desired action. (In case you missed last week’s newsletter about trust, it’s here!)

It’s easy to say “why don’t we force more budget to the ones with the highest ROAS” or “let’s turn off the ads getting the most spend so the system will spend more on those ‘better’ ads” but in reality, it’s unlikely that those ads would be able to keep that performance as they get more spend.

Turning off a top spending ad because it has high costs or low ROAS can be extremely detrimental. It’s likely getting the most spend because it’s able to scale more than your other ads.

This doesn’t even take into account the possibility that users could see one ad and later click and convert due to another ad (even without manually retargeting it to do so, the system can automatically do it).

Do you use Waze or Google Maps? If you do, you’ve probably gotten stuck in a traffic jam that wasn’t there when you started your ride. They make predictions about which routes can get you to your destination quickest based on current conditions and predicted future conditions (based on historical data), but driving takes time, and sometimes unpredicted traffic can build up. That’s the problem with predictions.

This can happen with optimized ads too. The system interprets signals that cause it to predict one way, but other factors might impact the relevance.

One major potential factor could be negative comments. If an ad is chugging along and then suddenly gets a bunch of negative comments, the system (probably) doesn’t understand the negative comments, but users can read the negative comments that might deter them from clicking or buying, causing less people to buy and performance to dip, but the system might not change it’s prediction yet until it gets more data, which takes time.

If you can stomach it, you’re often better waiting and letting the system figure it out, but if things are reeeeally clearly bad, you might need to interfere. I’d recommend doing this as little as possible. This is the exception, not the rule.

Focus on how the system spends your budget and look at overall high-level CPA or ROAS rather than specific lower-level CPAs or ROAS. The goal is to improve overall performance.

Let the system do it’s thing, and study the data breakdowns to better understand how and why your ads are performing the way they are. Don’t manually interfere. The system is usually very accurate, especially with the more conversion data you can give it. If you see ads are getting a lot of delivery on IG Reels or a certain age range, that does not mean you should separate out a specific ad set to target that more, it’s just a sign showing you where your ads are most relevant and competitive. Same goes the other way, if you think your ad should be spending more on a certain placement, don’t force budget to it, it will very likely perform poorly.

In order to scale effectively, you need creative diversity. Create a wider variety of ads, don’t always worry about making the top new ad in the account. Different ads that resonate with different audiences will have different results and get delivery to different users. This is a feature, not a bug.

If you need an additional ongoing source of a variety of performance creative, Adcrate.co is a fantastic creative partner that’s like having an entire additional creative team producing high-performance ad creative in your account without you needing to think much about it. I advise and own a piece of Adcrate, please reply to this email “Adcrate” if you’d like to know more or if you’d like an intro!

Speaking of feeding the system a variety of great ads:

You’ll be able to use these ugly ad templates immediately and get them live in your account in minutes.

If you want more than just my ugly ads, you can also check out templates from other experts like Dara Denney, Ashvin Melwani, Sarah Levinger, Rabah Rahil, and Nick Shackelford.

Use my code Barry10 to get 10% off!

If the breakdown effect is still something you or your team is struggling with or confused by, please book a consulting call with me and I’d be happy to dig into it with you.

If you’ve made it this far, you deserve something. Yes, even if you simply skipped all the way down here, you cheater.

For every 10 subscribers that reply to this email I’ll choose one person to get a free Make ugly ads hatt or ad account audit template.

Simply respond with which you’d want orrr I’d really appreciate it if you could also add any feedback about this newsletter! I’d love to hear from you, especially if there’s anything you think I could do better here. I usually respond to every email!

I hope you found this valuable!

Lastly, I’d appreciate if you could share this newsletter with anyone you think might appreciate learning more about the breakdown effect!

Hott regards,
Barry Hott