Dubious Data and Murky Measurements

Eaton’s Special Nickel-Cased Pocket Watch, 1917

“A man with a watch knows what time it is. A man with two watches is never sure.”

—Segal’s Law

Can You Show Me How To Do Data Analysis?

Recently, a person I worked with at a previous job contacted me to ask if she could take me to lunch so I could explain Google Analytics to her.  I was happy to help, but also wanted to help set expectations.  I pointed out that there were thousands of different reports that were possible in GA, and that was even before doing segmenting of visitors, comparing different time periods, and other more advanced ways of looking at the data.

The same week, a client asked me to show him how to interpret a report on Google Analytics.  That one report turned into almost an hour of conversation as we discussed some of the nuances that might be going on in the background behind the chart we were looking at.

Those conversations got me thinking about some of the things that seem obvious to me about looking at website data that don’t necessarily jump out at entrepreneurs who are just getting started marketing online.   One of those things is that the data is rarely clear cut.  Allow me to explain…

A Simple Example of Problematic Data

I track “reach” for my own social media marketing (among other things).  That is, how many times has someone seen something from my brand on social media.  I use Google+, Twitter, and Facebook, and all three have a way of showing what the reach is for your business account.  Easy, right?  Just look up the data for each site, compare them, and then I know how much reach I had on each site.

Au contraire, mon ami!

I bet at this point you expect me to talk about how each site measures reach a little differently and so they can’t be compared, right?  But no!  It’s even more complicated than that.

When I go to Google My Business (the dashboard behind the G+ business page) for my tea-education brand Tea Geek, this is what I see:

Total Views on Google My Business

Easy enough.  My all-time views are 313,846.

But hold on.  Before the site transitioned to the “new” Google+ format, you could display your views on your profile itself.  I switched back to the “classic” view and saw this displayed on the page:
Total Views in the Classic Google Plus version

My all-time views are 474,236.  That’s basically 50% higher.

Let that sink in for a moment.  In one place, Google says my all-time views are 313,846…and in another Google says it’s 474,236.  How can the same source (Google) give such wildly different numbers for the same thing?

Are they the same number…?

Okay, so the first thing I think is that they measure two different things.  I dig into the help files.

The total views number (313k), according to the Google My Business help files, includes how many times I’ve appeared on:

  • Google Maps
  • Google Search
  • Maps for mobile
  • Google+ page (as in, my business “profile”)
  • Other people’s streams (my individual Google+ posts…but it doesn’t count ones that might show up in Gmail)

It also counts views of photos I’ve posted (which might show up in Google+, Search, Image Search, Maps, and other locations).

Okay, and the other, larger number?  It’s supposedly just views of my profile and its contents.  So that would still be my G+ posts, page views, and photos, but probably not Maps, Search, and so forth.  This measure seems to be just looking at views within Google+ itself.  But it’s the bigger number at 474k.

How can it be that looking at a subset of locations returns a bigger number?  As I write this, I have to think it has to do with the time frame.

According to the help files, the smaller number covers the time period from October 1st, 2012 to the present.  I can’t find any data on when the “classic G+” profile number started counting.  However, Google first announced business pages on the social site on 7 Nov 2011.  I created my business page at some point between then and 17 Dec, 2011, when CircleCount.com added the page (9 followers!)

In other words, it might be that the profile number is larger because it started counting almost a year earlier…a year in which Google+ was new, and when business pages as a feature was new, and people were trying everything out and seeing what was available.  Then as the looky-loos stopped visiting and the novelty of business pages faded, the Google My Business count started.

At least, that’s my best guess.

Back to the analysis…

The real takeaway here is that analysis isn’t always a clear-cut world where every number is accurate and describes what you think it describes.  And that’s why looking at your website numbers isn’t straightforward and makes analysis a complicated endeavor.  Many measurements are mildly murky.  There’s a whole layer of knowing where inaccuracies might be coming from.  Or watching out for inconsistencies from one source to another (or in this case, from the same source).

Sometimes I’ll send out things you can check for on your own website in my monthly mailing list.  For example, I recently talked about how you can see if spammers have gotten some dubious data into your Google Analytics.  In fact, “data hygiene” is its own area of work, where errors and duplicates and other data problems get cleaned up.  So there are some simple, basic things you can do.  And then there’s the other stuff.

Of course, that’s what I do.  So if this sounds like something you don’t want to do, or seems stressful, or brings up questions in your mind, get in contact with me and maybe I can help out.

Have you run into dubious data or murky measurements?  What impact did it have?  How did you deal with it?  Let me know in the comments!

Image credit:  Wikimedia Commons

Note:  Google says it will soon be defaulting to the “new” version of Google+ for all users, so who knows how long the “classic” version will be available.  

Posted by Michael J. Coffey  |  0 Comment  |  in Analysis & Testing

About Michael J. Coffey

Michael started learning about online marketing as the web store manager for a scrappy little game retailer during the "dot com bubble" of the 1990s. Since then he's helped fitness companies, tea wholesalers and retailers, lawyers, clothing designers, restaurateurs, and entrepreneurs in many other fields. In his spare time he drinks very high quality tea, writes letters with a fountain pen, and reads literature.

Post a Comment

Your email address will not be published. Required fields are marked *

*

  • Stay Connected