Site icon Talking About Everything

Facebook Metrics Don’t Matter

Facebook has 1.2 Billion 2 Billion Active Monthly Users.

Facebook is Amazing

Facebook literally is the most perfect marketing system ever created.

But – It doesn’t work the way you think it works – Harry Hawk
I’m going to give you some examples of how I use Facebook

In a rush? No Budget?

Here is the perfect solution to getting an effective campaign running:

Put all your money in a metal bucket and burn it

You will save a lot of wasted effort and wasted time — use time “saved” to earn enough money to properly implement tracking and monitoring.

Common Facebook Mistakes

I’m going to point out some common Facebook mistakes, and then I’m going to drill down into a specific example and show why Facebook Metrics Don’t Matter.


My friend and podcast guest Dennis Yu (Blitzmetrics) has created a methodology for creating consistent results through multiple ad channels including Facebook. He has bought more than $1 Billion dollars in ads.

The way he plans and buys ads today isn’t exactly the same as how he bought ads a year ago, or three years ago. – Harry Hawk

It’s easy to look at something and think you understand it.

Its easy to look at Facebook ads and think it is more or less like buying Pizza.

One of the biggest problems is You; you may not be as good as you think you are. Just ask David Dunning & Justin Kruger – Harry Hawk

Dunning-Kruger Effect

People think they are better at things, and know more about things than they actually do. This is known as the “Dunning-Kruger effect.”

People tend to be blissfully unaware of their incompetence. (Dunning, 2003)

It’s you, it’s you, it’s all for you

Facebook is a complex system of real people & actions, bots, algorithms, databases and opaque systems – Harry Hawk

Dunning (2003) noted people base their perceptions of their performance…, on their preconceived notions about their skills, rather than correlate their performance objectively.

I’ve been researching ad and media systems exactly like Facebook since 1987 —  that doesn’t mean I understand any feature unless I test myself – Harry Hawk

Conceptual Mistakes

There are many ways to make a mistake buying ads on Facebook.

Some are easy to fix, and some are hard. For me, the hardest to “FIX” are conceptual misunderstandings.

Using Pizza as “proxy” for Facebook advertising, here are five common mistakes.

5 Mistakes People Make Marketing on Facebook

  1. They assume it works like ordering a pizza
  2. They act like they are the only one ordering pizza
  3. Think imagine the same person is making their pizza every time
  4. They get blinded by Numbers (like counting the pieces of pepperoni on their pie)
  5. They forgot Plato – they believe Pizza is real rather than a reflection of reality

Let’s examine these Five Mistakes

It’s not like ordering Pizza

When you order a pizza:

People “think” Facebook ad choices are like a menu — and “think” they can order anything they see.

They get upset when a targeted audience doesn’t buy whatever they are selling.

People don’t understand buying ads isn’t like buying a Pizza – Harry Hawk

Facebook knows a lot about people. Just ask Academic Paul Dehaye

When you order a Pizza you can get every topping or just 1

When selecting audiences on Facebook it is easy to select an audience that is either too big or too small – Harry

Some shops only make 1 size.

The size and selection of the audience(s) on Facebook maybe more important than the size of the budget or length of the campaign

I see folks pick an “amazing” – Kitchen Sink audience of Millions* and then spend enough $$ to reach less than a few thousand of them, some times less than a few hundred!

I see folks picking a precise audience of a 1,001 and then spend enough $$ to reach 10k people on a single ad, in a few days

You wouldn’t buy order 100 Pizzas if you can only afford one.

On Facebook you have to:

You are not the only one ordering Pizza

You might not think you are the only one ordering Pizza but by looking around the dinning room, and watching the pizzaiolos you can guess how many orders are ahead of yours.

At great Pizza joints like Luccilli’s they don’t even take phone orders

On Facebook you will have no idea of how many brands are:

On Facebook you will have no idea of how many people/users are:

You also don’t know how many dozens or or 100’s or 1000’s ads they might see in a day, week, or year.

At a great pizza shop you can guess how many people are ordering, on Facebook you can’t tell how many people are ordering ads or watching them.

People buy ads on Facebook like they are the only brand buying

They create ads like everyone who views an ad, will study it carefully – Harry Hawk

While some people will tell you they never ever look at ads, the reality is most people watch some and ignore some.

Here is a video I created, to target a very limited audience, with a limited budget, which was designed to appeal to that audience. Watch it now; or view later (all the links are repeated at the bottom of this page).

There isn’t just one Pizza maker

As I have mentioned, the greatest pizzaiolos in New York are the only person who makes pizza in that shop; if they are sick or on vacation the shop is closed.

People think about the Facebook ad system like there is one person behind a small counter making all the Pizza, 1 by 1 – Harry

The reality is on Facebook, they are always testing and trying out new ideas, new algorithms and new systems.

Each different method can have different options and different impacts. Choices you make about objectives, audiences, and budgets can allow your ad to be treated differently than others

I’ve bought Facebook ads targeting folks in more than 75 countries – and in over 35 different US media markets, and in over 200 different US zip codes – Harry Hawk

You can get consistently predictable results on Facebook.

That doesn’t mean a small sample buy on a holiday weekend will run the same way during a busy week day.

Don’t get fooled by the numbers

Every slice of pepperoni pizza has pieces of pepperoni on it.

Counting the pieces of pepperoni isn’t going to help pick the best slice – Harry Hawk

You may love pepperoni and want as much as you can get.

If you are given a choice of a 1/2 dozen different slices from different pizzaiolos – don’t let the numbers confuse you.

Size maybe more important than how many

Decisions. Decisions. Decisions.

There are many ways to make a good or bad decision. – Harry Hawk

Some business folks love playing poker because the information isn’t perfect, so part of the decision set is based on gut instinct (subjective), and some of it quantitative (you know how many cards there are, how many players, and you may even remember which cards have been played, etc.).

A large paid social campaign for a SAAS business will require multiple rounds of iteration and optimization based on statistically significant inferences  – Harry Hawk

Your $100 “Boosted” post has ZERO decisions that can be made by statically significant analysis of the data – Harry Hawk

It’s worth noting large companies, with amazing data containing “the right answer,” they often (mistakenly) ignore that answer. Just ask Claudiu Maraiu from

When you are using systems like Facebook’s Ad Manager and there are very important looking numbers (that actually are important), it’s easy to use those numbers to make very bad choices.

I explain below why the ad with the highest cost might offer the lowest price – Harry Hawk

My friend Claudiu is found of noting that when you look at, collect and analyze data, the questions you ask can be more important than the data itself.

People pick “winning” ads based on the click through rate  – the click through rate (CTR) — by itself never matters – Harry Hawk

Setting, picking, altering or avoiding ads or buys because of the cost per thousand (CPM), or the cost per click (CPC), or the CTR probably means you are making a bad decision.

Blame Dunning & Kruger but chances are if you make your decisions by the numbers in the Facebook dashboard, you are making things worse, not better – Harry Hawk

If you are broke and hungry buying the largest slice of pizza you can for $1 maybe the best choice. If you are looking for the most amazing Pizza and you are trying to buy a slice — you may have already lost because  Mark Iacono doesn’t sell slices!

Chances are, especially in your first few Facebook buys, you are only going to be making subjective decisions. E.g., based on your gut.

When you are making subjective decisions — don’t let the numbers FOOL you into thinking you are making a quantitative decision.

The Pizza isn’t real – it’s a reflection of reality

After a warm summer night in South Brooklyn eating an amazing pie from Lucali’s – it may seem like it it was just a dream.

The Platonic idea that what we observe in life is only a reflection (a shadow) cast upon a cave wall — may well reflect the quantum nature of the universe and probability that probability is one of the few certainties in life.

Let’s hope that all of your Pizza adventures are real and really good.

But if you are buying ads on Facebook (or anywhere else), the reality is there are few truths, and very little absolutes.

If you have a bunch of ads, blog posts and videos, on different systems and networks, and you are driving traffic to a landing page, and then your site
— every single bit of data you see is a dashboard is an illusion.

Some of the data is real, you just don’t know which – Harry Hawk

Looking at the data above a “typical” human would think:

The $27.39 cost per thousand (CPM) is way more expensive than the $12.14 CPM.

That is why they include Cost Per Result — its’ results that matter

Is the $1.64 cost per result is better than the $4.14?

It is impossible to tell from this data – making a decision based on this data would be the wrong choice – Harry Hawk

What matters is not how many clicks Facebook records but how many folks actually landed on the site.

Ultimately what matters which ads and messages produce active members, sales or other opportunities

Each System is Unique

Each system/network is going to count users and clicks and ads served using different methods and different code.

Those system will work differently with different devices, operating systems and be impacted by ad blockers, slow networks and user who are quick with their mouse clicks.

Follow the money it’s real – Kimberly Glazer (audio)

The numbers you are seeing may or may not be real, but ad fraud is real – bots purposely acting like users, may click on random on ads so they look “real.”

Who knows maybe bots are programmed to return every few days to make it seem like they are interested! – Harry Hawk

Important things to remember are:

Let’s Review
The Big Facebook Ad Buying Mistakes

  1. Buying ads is not like buying Pizza
  2. Don’t think you are the only one buying/bidding
  3. One buy may use slightly different tech and options from another
  4. Don’t count pepperoni pieces
  5. Your numbers and data are merely a reflection of reality

Here are some real world examples

Some good humored tales from several clients I have been helping or coaching with their paid social ad buying – Harry

Something must be broken

A client turns on a bunch of new creative – the next day it seemed like 191 visits to the landing page occurred.

I told them it was fake traffic (three times in 12 hours).

I said, “Maybe a few clicks from real folks, and few from the team testing the ads,” but I was certain I said, “most, probably nearly all of the traffic was from bots.”

I told them everything is fine.

The ads may or may not work, but it’s too early to tell.

Trust me, the landing page didn’t stop working.

The traffic didn’t just come from no where! It just doesn’t add up

The traffic literally came from no where. It didn’t exist, ignore it.

What happened?

It was fairly clear to me. At the precise time that the ads were submitted: several different campaigns, (and several ad sets) in total about 50 ads where turned on and off a few times by the client.

100% Legit:

Each of these actions triggered robotic testing of the link(s), triggering page views on a landing page, which were largely but not precisely confirmed by Google Analytics.

I did I know I was right?

Because of Dunning & Kruger I’m rarely ever given to absolute certainties; I was fairly sure.

I wasn’t ever able to convince them that something wasn’t broken.

For my client it seemed that the ads kicked-off amazingly well driving a huge amount of visits, and then stopped performing.

For them, the numbers proved something was broken.

Nothing was broken, there wasn’t any real traffic — the campaign was fine!

They were focused on complex scenarios like pre Copernican astronomers imagining dics, spheres and celestial clock-works to explain the orbits of planets and the motion of stars

In truth I wasn’t fully convinced myself until I saw it happen again – Harry Hawk

We Improved the Creative

Nestled into the multi-week long witch-hunt for the cause of the disappearing traffic was the certainty that this round of creative was better and more effective the previous round.

They had previously run limited creative with a high frequency to less than 1000 people

We were now targeting several million people – Harry Hawk

Each time they switched creative they killed the prior campaign and started up a new one. It’s an effective way to isolate the budget.

They went through several campaigns each one, targeting the same exact 1000 people – Harry Hawk

I don’t have any proof, this is a gut statement, but in my gut — it’s better to keep a campaign running and let the Facebook algorithm learn from your efforts and help optimize it.

Was it better?

Lost in the belief that the current creative was “better” — was an over fixation of the lower cost per click of the previous campaign.

Blinded by Science

Without running the first batch of ads long enough to see if people signed up, and become active members of the site (e.g., a good customer), it’s impossible to know if the ads were “working” or “not working”

If you don’t know if the first batch of ads were working – to attract good customers – there isn’t anything to compare the second batch to… except numbers.

Ignore The Numbers

The prior efforts had solid 20% click through rate from the landing page to the main site.

It was hard to explain that showing several different ads (over several weeks) at very high frequency to a narrow audience (super highly targeted)  –  couldn’t compare with running different ads to a different cohort, at different time/date, at very low frequency (because of the budget).

  • Showing new creative to previously targeted audiences is a close approximation to a funnel system, where a series of messages are shown as a message chain.
  • Each positive response is “rewarded” with a new ad or video until they convert

The initial ads may have worked “better” for a variety of reasons — including this “funnel” effect.

It’s also possible the first sets of ads didn’t work at all

They were convinced that the new ads would work better because they were better.. – Harry Hawk

Using gut decision making they made improvements.. but no focus groups, no quantitative user testing (things I don’t often recommend).

But just because they think the new ads were better (I thought so too) – doesn’t mean they actually are better!

One of the problems with the first few efforts was that the tracking and measurement wasn’t in place.

They didn’t know what was happening after the landing page – Harry Hawk

Their system was a WIP with a solid database but no easy to use report generator – the only way to tell if any of those early visitor became members was to pull their developers away from building the system.

They were upset that the cost per click went up about $2 a visit and the conversion rate on the landing page went down from 20% to 12%.

It’s hard to argue with numbers… which is why I often say to avoid them.

That is avoid the ones that clearly don’t matter. It’s hard to tell which is which!

Which numbers matter?

In this case the numbers of folks going from the landing page to the main site mattered, but what really mattered was:

Many ways to make a decision

It may well have been a great decision to keep with the initial audience and keep hitting them with high frequency creative, changing it every few days.

The small audience might have been the most affordable way to attract a few dozen or even a hundred users for testing

They Had Goals

In hindsight I may well have done better for them by coaching them to pick a smaller cohort of several hundred thousand people, rather than a larger multi-million person cohort.

  • That was one of the options I gave them

Our $600 Test

Using a cohort of about 5 million women and 4 million men – is way to big for a $600 budget.

It is a cohort that they can stick with for a few months; as they allocate more $$ for this effort, they can “work there way through” this cohort.

Men Are Bad News

The worst part of this test was that men didn’t join. They had previously targeted women because their research suggested they would more likely use the service.

12k impressions – no Men signed-up

There wasn’t the time or budget to re-target those men who hit the landing page with additional creative – there will be next time

Men and Women in this test were shown the same ads

Women Work Hard

I grew up in a family of hard working women; a Grandma who started her own business and a Mom who was a self starter.

For this client it is clear that Women are the right target for the short term.

Video Testing

The client to their credit created several short videos which we did run along with the standard creative. None of those ads directly converted, which is often the case.

The results?

Looking at the numbers from Facebook, the cost for sending each women from Facebook to the landing page was about $4.11 — significantly more than the previous effort.

Do we believe Facebook?

The bottom line is that any system including Facebook’s only knows what it can see and observe. It doesn’t know what happens after any hand-off.

An exception is using conversion tracking which we couldn’t do at this point

Even with conversion tracking, eventually there is always hand-off – Harry

What Actually happened?

This is an actual chart from the campaign i’m discussing. You may recall seeing in above.

Remember #’s don’t matter — what matters is the Cost Per Action – CPA

CPM, CPC, CTR don’t matter – Harry Hawk

What Really Happened?

Of all the ads that ran, only a few of them produced actual subscribed members.

1 Single Point of Truth 

Since systems are not going to agree, it’s important that there be a single point of truth. In the following example we used the cost spent on Facebook, and the results observed by Google Analytics.
You could say GA was our Oracle of Truth
Some companies use their own database or systems like MixPanel, Kissmetrics, or even Innertrends as their “point of truth” – Harry Hawk
Whereas Facebook has a price about $4.11 per visit to the landing page.


We observe a more reasonable $1.99; a $2.12 savings.


That’s even better than what they observed for their first effort; about $1.50 less.

Ignoring Men for Now

That $1.99 figure is the blended cost for men and women. Since women are the focus, for now, the breakout figure for Women only clicks is $2.28 per visit.

Why was the Google data showing better results than Facebook’s?

In my experience Facebook’s data is always off, sometimes it’s too low, and other times it’s too high.

Don’t forget Organic Reach – Harry Hawk

Some people think the only objective of using paid media on Facebook is to gain paid traffic. The reality is that with any “good” paid effort organic traffic comes along for the ride.

The difference in reported clicks is likely organic “pass-through”

How did we figure out Men vs. Women?

We could have just one one ad “set” and let Google Analytics (GA) sort it out later.

Facebook seems better at Demographics since some people are comfortable telling Facebook their gender identity.

We ran one campaign for men and one for women. Because we used different UTM tagging, we would tell in Google Analytics who was who, how many times they visited, etc.

A limited budget demands a focus on a limited target: Women – Harry Hawk

Cost Per Action

So while you can call is a CPC – I like to think about each stage as an effort to get a specific action.

  • CPC Landing Page $2.28 (Women)
  • $1.52 (Men)
  • $1.99 (both)
Which is why
You need one point of truth; MixPanel or KissMetrics (some day); today GA.


The chart above, you have seen it a few times now, only represents women.

One of the truly confusing issues is inside of Facebook the numbers are a siren’s call.  – Harry Hawk

The ad with the highest cost, was also the ad with the most conversions. That means it may have been the most effective ad.

Sorry, it’s too early to say – Harry Hawk

These numbers are great… but statically speaking it’s too hard to tell which one(s) are the winners; we can guess but we can’t be certain.

The one that cost the most, and generated the most users, also cost the most per user – if they all become “good” customers we might have a winner.

Numbers are important but only when you are asking the right question, and you have enough data to really know what is going on.
Ignore that call of the numbers – Harry Hawk
Three ads “worked”
All of the ads worked, but only three of them produced members.
With time, effort, retargeting and refinement, some of those who haven’t converted will do so in the future.

Cost Per Member?

My client doesn’t want me to reveal which ads worked, and the current cost of signing up a member.
I can confirm, when I looked through the data, and isolated just the $$ spent on women, I learned:
 The actual cost was about 1/4 of the price they first assumed
Cost per landing page visit
Women’s Funnel
20,101 Impressions CPA (reach) $0.0192
15,580 Unique Women CPA (unique reach) $0.025
133 Total Clicks = CPA $2.90
97 LINK clicks = CPA $3.98 | reported by Facebook*
170 Landed = CPA $2.28 actual CPC
166 Unique Landing = CPA $2.32
* earlier in the campaign this was $4.11 – it’s been getting more affordable over time. This is also something that typically happens but not always.

Digital Plumbing – Tracking & Monitoring

The ad that generated the most traffic cost the most.

Don’t forget your digital plumbing

That is neither good nor bad. Over time the cost will likely be reduce, and with optimization I expect the cost to drop between 1/2 and 1/8.

Today the cohorts are too small – Harry

In the future, when there are 10,000s of folks a week visiting the site, the real prize, and the real goal will be optimizing the effort to produce “good” customers.

How do you tell? Who is a “Good” customer?

Like most SAAS sites, it’s hard to tell now, but in time as the produce improves it’s “fitness” and as we get better at targeting new members, and converting them into active users there will be time when we can say which ads are winning at finding great customers.


I promised to put here all of the links mentioned in the article. Here they are (with a few extra (and some notes)).

Here is the link to the video showing how I created an ad to reach and target a very limited audience, with a limited budget:

Get your digital plumbing in place

Academic Paul Dehaye

Inner Trends (link)

Lucali Pizza (link)

Blitzmetrics (link)

Who is Dennis Yu (link)
Exit mobile version