Digital Design Blog

The Data Source Dilemma: Can Anyone Make The Numbers Add Up?

John Battelle has a fascinating interview with ComScore founder and Chairman Gian Fulgoni on his Searchblog today. The majority of the interview focuses on Fulgoni reacting to Google’s entry into the measurement market with their AdPlanner product, but I found this bit especially interesting regarding the state of web analytics today:

Battelle: In your comment on the Searchblog post noting Google’s Ad Planner, you noted a discrepancies between publisher’s server logs and Google’s numbers. Can you say more? Why is this?

Fulgoni: I suspect the main reason is that traffic numbers from server log data are inflated because of cookie deletion whereas panel metrics don’t rely on cookies and so aren’t affected by cookie deletion. As an example, Google Ad Planner shows mlb.com as having 9 million UVs in a month. comScore shows mlb.com as having 11.9 million UVs and mlb.com themselves have claimed they get 19 million UVs based on their server logs. Separately, I’ve noted comments on the blogosphere from several site operators saying that their Google Analytic UVs are twice as high as their Google Trends UV numbers.

I would argue that this issue transcends the spat between Google and ComScore and gets to the heart of the dilemma facing marketers today. Namely, how do you measure and analyze site activity when there is so much discrepancy from different data sources?

This frustrates our own clients to no end. The scenario that Fulgoni lays out for MLB.com is indicitive of the problem for everyone in the space — it’s so difficult to true up all the possible measurement tools into one common view. For example, a client may have Omniture for site traffic, ComScore for panel/meter data, Nielsen, plus offerings from Compete, Quantcast and even more. Correlating all of that is a manual process and it’s a beast.

I’m not a fan of arguing that one particular data source is the best (e.g. ComScore vs. Google), as each has merits in its own right, but until we have an automated way to true-up the data (one that doesn’t involve a team of whipsmart analysts) and get a better — easier — picture of consumer/customer activity, this will continue to be a sore point.

Read the full interview here.