Getting to a Gold Standard for Web Analytics Measurement
The announcement that the Interactive Advertising Bureau (IAB) is trying to drive toward a “gold standard for Web measurement” is good news in many ways. Theoretically, standards are a desirable goal, but implementing IAB’s standards won’t solve several fundamental challenges in Web analytics.
The Web Analytics Association (WAA) presented its own standards a few years ago, and many of the same problems remain. For one thing, there are still conflicting sets of numbers offered by different vendors and software packages. That means executives can’t get one clear set of trustworthy numbers to make investment decisions. For another, agencies and media companies often exploit the gaps and variances in trying to negotiate the most favorable rates when buying or selling ads.
Why is this? One reason is that different online universes will always be measured differently. What matters to huge media companies and financial services organizations differs from what matters to small local businesses, and so each measures its audience differently.
More fundamentally, the disconnect comes from the different types of categorization and tracking that are prevalent today. comScore and Nielsen are two popular audience measurement firms that compare user traffic across different companies’ sites. Each has its own dictionary of terms and measurement equations to gauge traffic. When it comes to internal measurement tools—where companies track the performance of their own sites—Adobe Omniture and IBM Coremetrics are two leading players, each with its own unique methodology. Even if all these companies use the same metrics (and the same definitions)—time spent, page views and unique visitors—the data is collected differently, and the numbers vary.
Let’s see how it plays out: imagine a user going from MSN to Google to ESPN to MSN to Travelocity to MSN, across a 30-minute period. comScore measures time spent for each segment of that visit. If a total of five minutes was spent at MSN on the various legs of that visit, comScore would track it as five minutes, because it tracks all legs of that visit. Internal measurement tools see this activity differently. Adobe, which uses page tagging to track user behavior, would count it as one 30-minute visit to MSN. IAB’s standards won’t solve this problem; even if Adobe and comScore adopted the same standards for time spent, their collection methods, coupled with different universes of users, would still generate different data for identical usage patterns.
The different needs and preferences of different analytics stakeholders is another gap that standards will struggle to bridge. Generally speaking, the advertising and marketing community looks at comScore, while media organizations prefer Nielsen.
Organizations in other industries—banking, for example—will focus on internal tracking to quantify how long people spend on online banking pages. If they use external numbers at all, it would be for research or general competitive benchmarking.
While we applaud IAB trying to increase clarity through standards, we would like to see them lead a large-scale effort to educate industry stakeholders on the important differences between all the different approaches to measurement, as well as the specifics of request-based logging, rendering-based tagging and panel systems. In our experiences, these differences are not well understood; and in many cases stakeholders (especially agencies) like to play the different methods off against one another in an effort to secure more favorable ad rates.
In the meantime, we recommend that companies educate themselves by digging down into the currently available numbers, recognizing that there will be gaps and trying to understand the different uses for the different types of tools. We have seen several companies use audit teams or analytics centers of excellence to do competitive analysis across sites, educate internal stakeholders and “track the trackers”—that is, regularly analyzing how different properties are tracked, whether by internal measures or third-party measures.
By understanding how internal and external tracking numbers relate, you may find real insights into the performance of your online properties and opportunities to improve it. When consistent definitions of metrics are applied across all digital operations, you will begin to see patterns in the metrics, like internal and external numbers moving in the same direction. This is a valuable and actionable insight that can be used to frame ad rate negotiations or to optimize content categorization so it fits most appropriately within comScore and Nielsen dictionaries.
It’s easy to declare a standard definition of certain metrics, but we’ll still face inconsistent numbers unless everyone decides to have one system to track everyone and everything. That is an extremely unlikely prospect. Still, a more informed stakeholder community would help everybody speak the same language and understand the numbers more clearly—and that’s what Web analytics is all about: gaining insight from the numbers so you can take actions to meet business goals.
