The Wheat & Chaff of Search Analytics

Add Your Comments

Synopsis – With a variety of studies indicating that SEO is becoming an integral part of more and more online businesses, the use of search analytics to track and administer search optimization efforts is also growing. A recent study by Conductor.com (carried out of eConsultancy in December 2010) of trends in the use of technology among search marketers found that 42% identified SEO metrics as “much more important” as influencers of business strategy in the last year, while 36% stated they were “more important.”  Only 4% identified them as “somewhat less important.” These figures reflect the growing importance of SEO to business, especially given that almost half (48%) of respondents classed themselves as “basic” in terms of the level of SEO they practice — experience has taught them the value of SEO to their organization as a whole.

In his article, “The Wheat & Chaff of Search Analytics,” Andrew Spoeth discusses the problem of measuring the results of SEO in light of the different models of search attribution. The key is to figure out what is vitally important to know and what is just extra, “nice to know” information and then find the appropriate methods to track that data. Andrew identifies seven key steps to take to separate out that core information from the extra details that are not vital to understanding what is happening with your business.

The Wheat & Chaff of Search Analytics

You get the dreaded email from your boss. ”It’s been a while since we chatted about our paid search campaigns. Please give me an update. Are they delivering value?” How do you respond?

Do you:

a) Dig out and finally read that copy of Avinash Kaushik’s Web Analytics 2.0 that your boss gave you last December.

b) Give a colorful but vague answer which contains half a dozen buzz words including Return on Advertising Spend (ROAS) and Incremental? Do you:

c) Build a new dashboard, or

d) Defer the question to a colleague?

Humor aside, this is a question which most digital marketers have faced, or at least should be asking themselves how they would answer.

In 2010, search is a relatively well understood medium and regular line item in many marketer’s budgets. Businesses utilize paid and organic search marketing to meet their objectives, the most common being to increase brand awareness, sell products, and/or generate leads. And companies invest a great deal in search. According to a recent Forrester Research study on interactive marketing spend, investment in search marketing in the US is growing and will continue to grow to over $30bn by 2014. This is backed by surveys and research which tout search marketing as an effective tactical channel. When it comes to delivering results, senior marketers rate search ahead of other channels such as email and display advertising (Sapient Interactive “Smart Agency Survey”, June 2009).

What is, however, the true value of search in the marketing mix? How do you know when your search marketing programs are delivering a positive return on investment? Despite having more tools and more internal data than ever before, many marketers struggle when it comes to answering this fundamental question.

Search Attribution Models

A natural first step in answering the search ROI question is to look at attribution. Attribution models are designed to link the advertising investment of a specific channel, e.g. PPC, banner ads, etc. with the sales revenue it generates.

Search attribution models generally fall into one of three categories: First Click, Last Click, or Hybrid.

1.  First Click Attribution

First Click Attribution assumes that the very first online engagement is the one worth remembering, at least from an advertising investment point of view. Which ad started the prospect relationship?

2.  Last Click Attribution

With this attribution model, it’s the deal clincher that we remember. Even though the customer may have seen or clicked on other digital assets previously, it is the final click before the sale which gets the credit.

3.  Hybrid Attribution Models

First Click and Last Click models have recognized shortcomings. Hybrid models – which consider more than just a single touch point – spread the attribution across multiple engagements. These models examine the “influence potential” of each of the touch points to the final sale. Compared to single click attribution, hybrid models can also be exponentially more complex. They require a lot of data to build statistical significance and in a practical sense, are not as easy to implement.

Which model can best serve as the answer? In examining the practices of successful marketers, there isn’t one single formula which wins out. But the best marketers do have something in common. They take a approach to measurement which separates the core, relevant information from that which does not matter to their business. And they do so convincingly.

The Wheat and the Chaff

When a wheat plant matures and is ripe for harvesting, it contains both the valuable wheat grain and the inedible, scaly surrounding material called chaff. There is unfortunately a lot more chaff than wheat, and the process of separating the two – called threshing – is necessary to reap the nutritional benefits of the grain. Threshing has traditionally required intense labor, and more recently, has been made easier through modern technology.

With measurement, there are seven steps that help separate the analytics wheat from the analytics chaff. Following these steps lead to a clearer sense of measurement’s purpose, regardless of attribution model or business.

Steps to Measurement Success

1.  Decide on roles

The first step to successful measurement is having a clear and common understanding of roles and responsibilities. Which sources provide data and how are they assimilated? Who generates the report? Who analyzes the data, and who makes decisions based on the data?

Successful marketing leaders proactively set out to define these roles for their team. This is also important when there are vendors are involved. A search marketing agency should, for example, report on traffic, clicks, conversions and costs. Complementary to that, the client can provide data on number of leads, subscribers, bookings, etc.

2.  First ask questions, then look at the data

Data can be fascinating. We get lost in it as it can take us down many interesting paths. With more tools, more reports, more possible customer touch points, it becomes easy to lose focus. Take a business-centric approach to analyzing data. What is the business objective of measuring a given activity? If you can’t find one, then don’t report on that data. Capture it in case it’s needed in the future. But don’t waste time digging in numbers to find something which is intriguing but irrelevant.

3.  Ensure measurement is connected to today’s reality

Customer behavior can change over time. As an example, we look at a Fortune 500 company selling industrial supplies that has recently had to change the way it measures online transactions. A 30-day cookie was used to track traffic from advertising, traffic that would eventually lead to specific sales. Since the average sales cycle was well under a month, this tracking method worked well and allowed for proper attribution of sales revenue against advertising investment.

And then when the recession hit in the fall of 2008, the company noticed a shift in buying behavior. Customers began comparing prices to a greater degree and putting more consideration into their purchases. Sales started to take longer, with a significant amount falling outside the 30 day window. Thus, the company prudently changed their tracking methods to adjust to new realities.

4.  Step back and see the forest for the trees

Remember the Magic Eye(R) posters from the ’90s, the ones which reveal a 3D image if looked at from a distance? They are a great metaphor for data analysis. The images are made up of hundreds and thousands of little points, points of various color, shape and size. Individually, the single points are meaningless. They appear as random noise. But put into the context of thousands of other points, a pattern emerges across the page, a pattern which reveals a three-dimensional picture to the viewer. The lesson for web analysts? Details are important, but their true importance is revealed when considered in context, i.e. the big picture.

5.  Tell a story

Data analysis lends itself to abstraction and can easily become disconnected from reality. Telling a story, a story based on an individual customer, is a great way to overcome this. A couple of years ago, when introducing marketing automation software at a B2B search agency, I crafted a persona named “Jackie” with this end in mind. I described how she went about engaging with a brand over a period of several months up to the point of purchase. The story was told from the customer’s point of view. Alongside the story, I provided examples of how the automation software measured and responded to her online interactions. The story was easily understood and effective at getting a common understanding amongst the marketing team.

6.  Build a model with placeholder numbers

For companies starting out with analytics, it may be difficult to know where to begin. Instead of digging head first into the data, start by building the type of report you’d like to see with dummy numbers. This type of report will likely be a reflection of what you already know about your customer buying behavior. By building the report, you have essentially asked the questions. And knowing the questions allows you to look for the answers in the data.

7. Find  out the How, then the How Much

Understand the process, then measure it. Successful marketers are great at getting to the core… the core of the right thing. First look wide, but then quickly narrow in on what truly needs to get measured. Measure the ‘wheat’ with precision and over time. Watch for trends and re-evaluate on a regular basis.

Follow these steps and you’ll be prepared if – or when – that dreaded email arrives. More importantly, you’ll be well on the way to a deeper  understanding of what the numbers your analytics programs collect really mean.

About the Author

In late 2010, I joined Marketo, the most exciting, successful marketing automation company in the world right now. At the company, I work amongst a very strong team of B2B marketers to help manage demand gen programs. Previously, I worked as the marketing director for Enquiro, one of North America’s leading search marketing agencies. In my role at Enquiro, a big part of my job included being in touch with the innovation and advances in our industry.

Add Your Comments

  • (will not be published)