Why Lady Gaga Shouldn’t Be In This Headline (Or: What It Means to be Data-Driven)

A recent post over at Web Analytics Demystified about the “myth of data-driven organization” generated a good bit of buzz in the field. Indeed, the comments section of the post is very interesting, and suggests that analytics practitioners are coming to terms of what it really means to be “data-driven.”

The debate boils down to the question of how dependent companies should be on raw data to make decisions. In the online marketing space, that might mean looking at performance data to find the right ad network or set of online properties for an ad campaign, tweaking promotional offers (e.g., buy one, get one free vs. blanket 30% off) and choosing new keywords. Within digital media, publishers are more likely to address questions around content placement and monetization strategies.

Some data purists seem to believe that data should dictate – and even automate – much business decision making. You choose offers, ad formats or content placement based purely on performance. Whatever attracts the most eyeballs wins – period and end of story. The only problem with that theory is that Lady Gaga, the cast of Jersey Shore and NFL scores would have to feature prominently in every page of a site. And if you’re not in the business of sharing celebrity news or providing sports content, well … you get the point.  (Actually, some decidedly non-celebrity media sites are quite comfortable using Lady Gaga for headlines.)

We don’t agree with this “bait and switch” practice, but sadly we see it all too often. The point is, Lady Gaga should not be in the headline of this post, but we expect at least a few clickthroughs based on our little trick! Particularly sad is the fact that some people seem committed to prove that this sort of trickery is productive use the “data-driven” concept, which sounds so worthwhile, as justification. “As a data-driven organization,” they seem to say, “we have to put Lady Gaga in our headlines.”

Obviously, this is not the way to go. Raw data is more prevalent than ever. But that doesn’t mean organizations should be driven by any data or seek to capture the most data. It’s about finding out which data is most important to overall goals or specific challenges, and then leveraging that particular set of data in the proper way.

Sure, top businesses and admired non-profits are strong quantitatively in how they measure their sales and operations. Yes, they have lots of great data, but they are also exceptional in qualitative terms. They have strong leadership to set the mission and inspire the entire organization. They use management and operational infrastructure to execute against core strategies. They establish cultures that enable their people and teams can perform at a high level. In fact, a “strategy-driven” or “culture-empowered” organization would beat a purely “data-driven” company any day of the week.

It takes a lot more than data to succeed, in other words. Companies must also have the skills, processes and resources to leverage it properly if they are to gain the potentially huge “data-driven” advantage over their peers.

It’s important to remember that even the most admired “data-driven” companies, like Amazon and Capital One, are far from perfect. If you’ve ever bought a baby gift for a friend or music for your teenage kids through Amazon, you know that many of the follow-on product recommendations are not exactly on the mark. That’s what happens when you business gets turned over to the algorithms.

Such “best practices” reflect the fact that there are still too many limitations when it comes to data collection and real-world analytics today. For one thing, data sets are never really complete. The next Lady Gaga-sized sensation, who will generate unprecedented numbers of tweets and likes sometime in the very near future, is on very few radar screens today.  (We’ll talk about predictive analytics another day.)

Similarly, web analytics today, with many different tracking and measurement methodologies, creates gaps and grey areas that must be analyzed and understood. As much data as we have, analytics pros recognize significant variations in how even relatively simple metrics (like page views) are defined. So if you’re going to be data-driven in our fragmented analytics environment, you have to ask yourself which data set you want to do the driving.

Strategic context may be an even more important issue, as several commenters to the original post mentioned. One argued for measurement concepts like “popularity quality” that would balance raw traffic measures with notions of strategic alignment. As with complex metrics like customer engagement, the analytics community has some work to do to nail down robust and statistically valid means to capture such concepts. In the meantime, companies must understand even simpler metrics – like raw traffic and impressions – within the context of their strategies. Different companies need different metrics because they have different strategies.

Lastly, the decision making and reporting culture have a role to play here. As data increases in value, it’s important that senior executives understand where it comes from and its various limitations. Reporting tools, formats and tempos must be structured for effective use and clear information sharing in ways that make sense for the business. In our experience with high-performing organizations, data analysts have a role to play in educating and counseling business stakeholders and decisions makers on the data itself.

In the end, those commenters who suggest that this is largely a semantics question have a point. Whether it’s more about being “data-informed” or “data-enabled” as opposed to “data-driven” may not matter that much. What matters is that online marketing, digital media and analytics teams must validate that they have the right data based on their unique business goals and objectives, and then use that data in the best ways to enable the achievement of those goals and objectives.

Not as exciting as Lady Gaga posts perhaps, but then again we’re more likely to click through on headlines about conversion rates and page tagging than on Lady Gaga!