Categories
Social Media

Problems analyzing Social Media

Introduction

 

Before you start, if you are curious... ¿Where is this website coming from?

Maybe you remember my old website, which was all in Spanish and had a few blog posts and Pages.

The site went down because of a malware problem, and I wasn’t able to easily restore all the pages and contents created.

At the beginning, I felt very frustrated, but I finally decided this was an opportunity to start over, with a more simple and purist design, and the sole purpose of start sharing content again, but from a completely different point of view: No more hacks or step by step SEO tutorials. If I happen to write this kind of content again, I believe it will be in third-party websites.

I just want to share some thoughts from a very strategic and personal point of view. In the long run, I believe this kind of articles will be more helpful and inspiring for my small but select audience.

This first post took a lot of thinking, a lot of reckoning, and writing back and forward; so I really hope you guys enjoy it.

Many years ago, I discussed with a friend that many forms of traditional advertising would end up disappearing because it was impossible to measure its impact.

How do you assess the impact of a magazine’s page, or an OOH creativity? It’s impossible to know how many people actually looked at it, much less how many of them liked it, understood it or felt engaged by it”.

My friend answered “classic advertising works by some sort of magic, but it works, and it will continue to work”.

picadilly circus ooh advertising
Piccadilly Circus advertising screens
cc-by-sa/2.0 – © Oast House Archive – geograph.org.uk/p/5633131

At the time, I thought digital marketing would end up absorbing most of the media investment because everything is measurable, traceable, and therefore optimizable. I didn’t think that this “investing on magic effects” would had a very long life ahead, and obviously I was wrong.

After a couple years working in advertising, out of the ‘strictly digital bubble’, it’s obvious to me that traditional advertising works. Especially if you combine different touchpoints in a smart integrated media strategy. You don’t have to measure the specific impact of each asset, “the circuit just works”.

However, what bugs me is that I have also stumbled into the suspicion that many times we are doing a pretty shitty job when “analyzing” the impact of digital campaigns

mr robot
There it is, that feeling scratching that part of my head. It bugs me.
Illustration by Topher McCulloch on Flicker

So yes…, of course we measure stuff. We do a lot of reporting, we try to pull out some learnings, but… do we actually know if a campaign was successful or not, and why? 

Do we actually measure how digital assets are contributing to our communication and marketing goals?

Unsettling as it might sound, my humble opinion is that most of the times we don’t. We just check stuff because it’s there, easy to check. But we don’t always learn much about it, sometimes we don’t even understand anything about it.

So this is a post about how we don’t get shit right when it comes to measure results on social media, and how can we improve that.

Problems when analyzing Social Media

To make it easy, I’ve made a list of 4 things that I feel many people is getting wrong about Social Media reporting and analysis.

These are not stone-engraved statements. Just the best way I found to put my ideas in order to make them easier to swallow. 

If you feel criticized because not only you do these things, but you believe they are well done, I will be happy to discuss any of them with you in the comments, in Twitter or LinkedIn

1.- Measuring everything for no reason

What do we measure, when it comes to measure Social Media performance?

Probably, most of us are checking the same metrics for every post: Reach, Views, Impressions, Clicks, Engagement Signals (Reactions, Likes, Shares, Comments…), Engagement Rate, Share of Voice, Click-through-Rate, Cost-per-Click, View-Through-Rate….

We tend to “measure everything” for all the assets we publish, and often enough that leads to an awkward outcome: we are not sure if that post worked well or not.

THINKING

Analyzing results is not about piling numbers because “they are there”. It is about investigating if the goal you were pursuing has been accomplished.

And to do so, you need to establish a clear objective for every asset from starters. Not once it is published, not when it is done: from the very beginning.

We should be trying to find out if a specific asset served its purpose. Checking all the numbers is not going to provide “more valuable information”. On the contrary, it can make you miss the point of what is important: if you are getting what you wanted from that post or not.


Example:

We are trying to raise Consideration for a product by displaying all its features in a 2 minute video.

¿What should I be checking? High VTRs (View-Through-Rate). Reach & views should be coming from the media plan, and Viral Reach is only interesting if you are reaching new users that could also become buyers.

Why should I care about getting a 100 or a 1.000 Likes (that could be coming from bots or fake users) if my VTR is dropping at second 10?

Nobody is seeing what we needed them to see, so probably nobody is considering to purchase the product, which was the whole point from the beginning. You need to find another way to tell these people what you wanted them to know from the start.

The prior goal was to show your target all the things the product can do. VTRs is the KPI that allows you to confirm that the audience was actually getting that message.


The goals we set must target human behaviors and reactions, not “numbers”.

KPIs are indicators to understand if we are inducing the human behaviors we expected. KPIs should be used as signals for comprehension, not as ultimate goals. Because no KPI is going to the store to purchase your product, humans will. 

Adding layers of noise to the analysis will easily end up clouding your judgement. “Look, Engagement Rate is high”. Yes, but we didn’t want people to be engaged. We wanted people to know more about the product, and that is not happening.

Many times we are so focused on producing new assets that we forget to specifically think what they are for, and that makes “analysis” impossible. How are you going to measure if something is working out if you never really thought what were you expecting?

When you set prior and specific goals to every piece, it gets really easy to decide which metrics you need to be checking afterwards.

Specific” is important. Don’t say “branding”. Branding how? Do you want to raise Awareness? Do you need to associate a brand to certain values? Are you trying to make your brand look trustworthy? Do you expect to create conversation from a specific niche of people around it? Don’t go just with “branding”

In every Briefing you should specify as clearly as you can what will be the purpose of that creativity, and the metrics you are going to use to check its success. 

Once you are properly monitoring these specific goals; you can still measure everything, if you feel that you are missing something. Just note that there are cases where certain interactions are just weird. Why would anybody “Like” a post about a dental insurance? Why would anybody “Share” a post about a suitcase cubic capacity? Of course, it happens. But, either these interactions are coming from hyper-active bots trying to hide their activity, company employers “trying to help” or from really weird users with weird motives. Whatever the scenario, why would we care? 

So, first learning:

Set a prior and specific goal to every creativity.

Base this goal on human behaviors, not numbers.

Monitor only those KPIs that really show if you are achieving that goal. Optimize on that.

Forget about the other stuff, it’s just a waste of time.

2.- Assuming interactions are good

Think about it this way: If we consider that an interaction symbolizes a person expressing an emotion, don’t you think it is pretty stupid to believe that all of them will be good?

Engagement signals might be bad

In Social Media, we love “Engagement”. Engagement is great. People loving our contents for the joy of our clients…

I am not a big fan of Engagement, and the main reason is that I don’t feel comfortable with how this metric is usually treated.

Merging interactions to “one KPI”:

Interactions are a very common KPI in Social Media. However, not all Interactions weight the same. This is very obvious, but still most marketers don’t take it into consideration when it comes to reporting.

Why do we count different Engagement Signals (Reactions / Comments / Shares) under the same umbrella? They suggest very different behavioral patterns.

When we calculate Engagement Rate, we usually go with Interations / Impressions x 100. Why do we feel comfortable labeling with the same number a post with a 1.000 Likes and 0 Comments than a post with a 200 Likes and 800 Comments? It is very clear that those contents are leading to very different reactions from the Community.

Assume the correlation “the higher, the better”:

It doesn’t make sense to count interactions without any context, and assume the correlation: “the higher the number, the better the results”.


Classic “High Engagement Mistake”: A post with tons of Comments gets spotted as a super top content, when it is actually filled with aggressive / negative comments towards the company, or simply pointing the content as bullshit.


“Comments” might not be good:

I have seen this a lot and it is really harmful. Not only you are not learning anything, (you don’t have a clue about what is making your Community angry), also you are foolishly assuming that everything is working out great. That could even mean  that you turn that post into benchmark, giving it more Paid Reach and posting more contents alike.

Not only you are not stopping the bleed, you are paying so angry people have a wider reach to complain.

Sentiment Analysis is the answer to that problem: export all the comments, filter the unrelated ones, divide them into Positive / Neutral / Negative, and pull out the issues that stand out in each category.

Of course, that is a lot of work and it is expensive, but it is the way to get valuable information. If you lack the resources to do so, don’t just assume that “everything said is necessarily good”. Assume nothing, and at least you won’t be making any serious mistakes.

what kills you

So if Social Media is important to you, and you can’t manage to have Sentiment Analysis, at least try to have good Community Management.

Good Community Management won’t be as sharp as Sentiment Analysis, but it will give you hints about what works and what doesn’t. Good CM is about spotting what stuff makes your community cheer, and what pisses them off; and tell the rest of the team so they can adapt the content strategy. 

Going through comments on Social Media is super-underrated. You can learn much more from randomly reading comments than going over reports about how you got “piles of likes, comments and shares” without any context.

Reading comments allow you to pull a lot of interesting insights from your CommunityYou might spot hints that can turn into smart business decisions after some research.


Example:

Once, I found out that the Frequency of a video-ad campaign was too high thanks to Comments. The campaign was having a super positive reception, but there was an increasing number of Comments from users complaining about how often the spot kept popping out.

Even some of the users who originally commented positively about the campaign were upset because they were seeing the ad too many times.

That was great input: By cutting down the Frequency we were saving money and we avoided burning the campaign for over-exposure.

There is NO WAY you can pick up on something that valuable just from checking cold out-of-context “Interactions Reports”.


Another very classic and harmful situation: Many times, users pick random posts to complain about something that bothers them about your brand. They don’t care about the actual content they are commenting on, they are just “complaining”.

It’s not only smart not to count them as “yeah, cool, Comments”, it’s smart to try to understand what is bothering them so much to keep spreading their moaning across all your posts. Maybe there is something that you should take into consideration for your content strategy, or even your marketing strategy.

I will take “a few good details” over a “vague overview” anytime. Read comments, guys. You will be surprised.

“Likes” and “Shares” might be tricky too:

You might be thinking “Yeah, OK, Comments are a tricky one…, but Likes and Shares are straight-good. There is no way around it”. 

Well… Not necessarily. Facebook we have six kinds of Reactions, and 2 of them can be interpreted as bad most of the times (Sad and Angry). It might feel very neat-peaky to spot that to rule out Facebook Likes, but I believe is pretty silly to consider that these two just “do not exist, and we will count them as good anyway”. Again, if you can’t manage to measure something properly, assume nothing and at least you “won’t be wrong”.

If you are now thinking “this dude is a real pain in the ass, you can’t take anything easily“. Do you remember the first point, when I said “check only those KPIs that actually matter”? Well… That.

Besides, as I pointed out before; sometimes it’s hard to give Likes much credit. I have seen videos filled with Likes, but their VTRs were massively dropping at 2%. People weren’t even watching the video, but they “Liked” it.

Of course, I am not benchmark for “regular user behavior”, but if I look at my own experience, I only “Like” things that I actually like very sporadically on Twitter and LinkedIn. Elsewhere, my Likes are just “political” (work) or “polite” (family, friends).

Many times, when I analyze posts from brands, the “Like count” reveals really weird behavioral patterns, which leads me to think that there are many other users out there that like stuff that they don’t “Like”, and “Like” stuff they don’t give a shit about. It’s kind of a “cheap metric” to me. Easy to get, but not very revealing on many occasions.

When it comes to Shares and ReTweets, yeah, those are usually good; but there is also the possibility that people is sharing a post to criticize your company.


Example:

I remember a benchmark where we found out that the top-standing post for a whole group of retailers was confirming that there had been a serious problem of food poisoning with certain products. The brand urged their clients not to consume those products and come by the store instead to give them back.

People was sharing that post to spread the word. It was a serious reputational crisis, filled with Shares. You could say “hey, this poisoning thing is doing pretty well in Social Media!“. I don’t think so.


Second learning:

Stop assuming Engagement is necessarily good and start looking into what is behind this interactions.

Try some Sentiment Analysis to turn Comments into valuable insights about your Community.

Engagement might be good, but not necessarily; and it can definitely not be important on certain cases.

3.- Comparing completely different assets by the same metrics

Judge by likes

Pretty much all the same metrics are available for every Social Media post: Likes, Comments, Shares, Impressions, Clicks, Reach…

We tend to include all those metric in our reports, because they are there, easy-for-grabs.

And since some of these metrics are public (Interactions), it gets easy to draw comparisons between our posts and our competitors.

We end up looking at reports that compare creativities with  nothing in common by the same KPIs, and we even pull rankings, using these metrics that most of the times haven’t any relation with the actual goals these creativities were created for.

When you say it like that, it sounds stupid; but it keeps happening. It amazes me, but there is people making a living on these reports that are an absolute nonsense.

Why would anybody want to compare by the same rules posts that are obviously from a very different nature? Why would I rank based on Engagement any kind of post? Of course there are posts that gather higher Engagement Rates, but that doesn’t mean they are over-performing other creativities that are not meant to accomplish that.

I could be beating every one of my competitors on Social Media and not having one “Top Engagement Post” if that is not on my strategic roadmap.

The problem with all this is that, despite being very obvious, we keep stumbling into these reports & rankings so often, that it ends up corrupting our thinking of what works and what doesn’t on Social Media. Specially, if the client is “buying” the approach. If you are not able to convince them that this is not the way to measure success, it creates a vicious circle where we start publishing posts to gather as much engagement as possible without any further intention; replicating the same structures, the same formats and the same tricks to push more and more Likes down the gutter.

We end up working towards the platform, not our clients. We post to get Likes, not to communicate something. And that is a total waste of time and money.

Likes out of business

I don’t want to point fingers, but if you work on Online Marketing or Advertising, I’m pretty sure you will be able to identify when brands are just posting to get Engagement.

What is the point? “Likes” are not going down the store to purchase anything. Likes are just there. If they reflect that your brand is connecting to their audience, awesome. If you are using silly games like “pick A or B”, and you are not building towards anything, I believe that you are wasting your time.

Third learning:

Stop comparing creativities that doesn’t share the same nature by metrics that are unrelated to their goals.

Don’t pull learnings out of “Top Posts reports” that are lacking every piece of context.

Focus on benchmarking how other players achieve the same goals you are trying to achieve, and leave all these silly rankings behind for good. 

Don’t post just to get Engagement without further intentions. 

4.- Missing on the bigger picture

Both agencies and clients usually work in SILOs. From my experience, Social Media is a field that often enough falls between departments. Who should be in charge of Social Media? Communication & Advertising Department (TV, Display, OOH…) or the Digital Department (Web, Newsletter, Corporative Blog…)? 

It is hard to integrate Social Media in a big company’s omnichannel strategy, because it can potentially play a role enhancing all the other touchpoints, and it is difficult to get its full potential if the responsibility lies only on one side of the table. 

Many times, Social Media work as bridges, connecting platforms where goals are achieved.

And how does this relate to our present matter, reporting on Social Media? The thing is that even if we are getting the right KPIs, we might be missing on what is really going on if we do not pay attention to the bigger picture.

I am going to use a very straight example to explain this point, and you can add all the layers of complexity when carrying this line of thinking back to your business.

Imagine that we have kicked a campaign to get new subscribers to our Newsletter: We have different creativities, to test which one works best; and we are monitoring all the right metrics: Frequency, CTR, CPC…

We are even checking how many of these users are actually subscribing, and we are getting a very healthy conversion rate. Ads are working, the landing page is working, everything is great; case closed.

However, if we start digging in a little bit we find out a very different situation: It turns out that since this campaign started, our rate of Opened Emails from our Subscribers List is getting lower and lower. And weekly, a great deal of members are unsubscribing. 

Despite we are managing to get new subscribers we are not really increasing our actual database, since most of the new contacts are not opening the emails or they are unsubscribing soon after getting listed.

At the end of the day, we are gaining very few new subscribers, to a very high cost: Ads running & being optimized, landing page running & being optimized, higher amount of weekly sent emails…

The Social Media campaign worked out, that’s what numbers will tell you; but from an end-to-end perspective you are not getting good results. You will need to look over the segmentations you are targeting, the Ad’s promise, the landing’s value proposition, the Email’s subject… try new strategies: maybe you need a different line of content for new Subscribers, or some kind of Welcome Emails before getting them into the regular Email flow… 

Whatever you decide to do, it is just not possible to find out without getting a higher perspective, beyond the sole Social Media performance.

Driving traffic that is poorly performing, getting leads that never turn into customers, impacting users out of our actual target groups… there is so many ways to be running successful campaigns in Social Media that are actually failing from the global perspective

Fourth learning:

When analyzing Social Media results, try to get enough data to reach an actual understanding whether if the end goals are being fulfilled. Don’t stop looking at the first step, even if you have prove that this first step is working out, since it can be leading to the wrong path.

Closure:

Ok, guys, this is it! I hope you enjoyed this first post.

Please, if so, do not hesitate to Share this post with whomever you think will like it or learn from it.

If you have something you might want to add or discuss, I will be very happy to do so in the blog Comments. Have you find yourself making any of these mistakes? (I have! Many times!), is there any other wrong practices you would like to talk about? I’m waiting to hear about them!

See you soon! Stay tuned!