Return on investment: three little words that strike fear into the heart of the average marketing director. At least that’s the case if the trade press is to be believed.

Terrorised by their finance director, perceived as flaky by procurement and rarely regarded as boardroom material by senior colleagues, the average marketing director doesn’t get a good press. Their key weakness is said to be the ability to quantify and justify their expenditure, a vital skill as marketing is often the largest discretionary spend in the company budget. While stereotypes do not apply in all cases, they do often hide a greater truth – that traditional marketing measures do not stand up to today’s scrutiny.

It’s no surprise that return on investment has risen up the marketing agenda in recent years. And with TV advertising often the largest part of that discretionary marketing spend: it’s an issue that’s vital for anyone considering using the medium to address.

Fortunately ROI is an area where TV can be fairly said to lead the marketing mix.

“You tend to see bigger measurable effects on TV than you do with small niche channels,” says Mark Greenstreet, managing director at Carat Insight. “TV has a lot of heritage of evaluation and normative data compared to other media.”

“Some people think that TV is a lot less measurable than other marketing, things like the internet and direct mail,” adds Les Binet, European director of DDB Matrix. “The one medium where you know most about ROI is TV.”

There’s more data and more public case studies on TV, combined with the fact the impact of TV advertising is such that the effects are often easier to detect. It’s also an area that’s had more scrutiny than most as TV is usually a key element of the Institute of Practitioners in Advertising’s Effectiveness papers.

The first issue for anyone considering a measurement of return on investment is to decide what sort of return the brand wants to measure.

In the past many proxies have been used for a fiscal return on investment, these can include awareness or consideration or other soft measures, but Les Binet, European Director of DDB Matrix, argues that business numbers are the only way. 

“When I talk about ROI I mean the effect on your sales and your profits. It’s very important to be very clear,” he says.

Ingrid Murray, managing director at Ninah Consulting, agrees:

“In theory it should be profit but a lot of marketing directors like to look at uplift in sales so they can compare like for like.”

However, it’s also worth considering the components that lead to the sales and the profit.

“Profit and sales are the end result of a wide range of factors,” says Dave Brennan, research and strategy director at Thinkbox. “A campaign that drives positive movement in brand equity scores, for example, would also have an impact on sales and may eventually be the most important driver for bottom line profitability, especially over a longer time period.”

Business measures

The benefit of using business measures is that in stripping out relative performance of channels such as advertising and all the different media channels, direct and PR, analysts need to find a measure that works across all of them.
“Probably a lot of marketing directors will measure their TV in terms of Millward Brown awareness and measure their DM in terms of response rates, that’s comparing apples and oranges,’ says Binet.

ROI has risen up the agenda because these traditional proxies for business return are no longer acceptable.

“The reason that ROI has become more into focus in the last 10 years has been the recognition that those measured proxies do not necessarily convert into the bottom line business,” says Greenstreet. “Marketers and other people in agencies have challenged some of the historical proxies.”

“The problem is that most people use lots of other proxy measures to measure the effectiveness of their advertising channels,” adds Binet. “None of these things is a measure of pay back and they can’t be compared.”

At the same time however, changes in the way marketing is carried out has made the evaluation process more complex. Ten to 15 years ago spend on an advertising campaign might be split 85% TV and 15% on a secondary medium such as press, outdoor or radio. Any positive impact could be broadly ascribed to TV as that was where most of the money had been spent.

“It’s become harder because so many campaigns have become multimedia, “ says Greenstreet. “Almost the going-in consideration is that a brand communication strategy will be multifaceted. In the past it would be TV plus support and the majority of the investment would be TV.”

Much of the challenge in today’s more complex world is disentangling the effect that TV played compared to other media.

“TV’s role in driving business success can be under-rated. Traditional measures such as awareness data don’t always take account of the way that the brain sub-consciously processes brand messages, such as through TV advertising, even at low levels of involvement or attention. These emotional associations, laid down by TV, are later triggered and amplified through contact with other media,” says Brennan.
So how can marketers ensure they are getting good value from their investment in TV? The traditional and most straight-forward way has been test and control, using a different media mix in one area to the one you use in another.

Test and control

“It’s quite a good idea to at least consider the possibility of doing some kind of controlled testing as part of your media plan. That might be a silent TV region, doing things at different times in different regions or different weights in different regions,” says Binet.

The problem with this approach is that while terrestrial channels do provide regional macros anyone using multichannel TV will automatically be targeting a national audience.

“It’s harder to run test and control as the terrestrial channels become less dominant,” says Jeremy Griffiths, effectiveness director at MediaCom, adding that dropping London from the schedule on cost grounds doesn’t always suffice as it’s not a typical region.

The other alternative is some kind of staggered media schedule.

“If you run different activity at different times and there’s clear separation between them then it’s easier to see the effects, run the TV first and outdoor later and direct mail later than that,” suggests Binet.

“You can start the radio two weeks after the TV and most of the time there’s enough variance in the national delivery,” says Griffiths. “We will always have a problem if three media run at once over the same time period starting and finishing on the same day.”

And with the rise of the integrated campaign where all media work together and often run at the same time that is making evaluation of the different elements including TV more difficult.

“There’s a trade off between integration and accountability. The more integrated a campaign becomes the less accountable it becomes,” says Binet. “It does raise the problem, because it’s much harder to see what’s working and what isn’t.”

And while the consensus is that integrated campaigns are indeed more powerful than non-integrated ones it’s vitally important to ensure the evaluation doesn’t distort the communications plan it’s designed to assess.

“We tend to avoid compromising the plan simply for the ability to get a better measurement of it,” say Griffiths.

Statistical tools

The tool that many agencies use to work out these conundrums is econometrics, a statistical tool designed to wheedle out the different impacts of each communications channel from TV to radio to PR and word of mouth.

“Over the years, the growth of econometric modelling has sought to convert advertising into financial business return and to a large extent does it very well. It’s a statistical approach that’s able to sort out the effect of TV advertising on sales from all the other things that might be affecting sales,” says Greenstreet.

“If you’re looking to prove something has worked previously you’d definitely use econometrics. If you’re looking to see if something will work in the future you’d look at something like predictive modelling of sales responses to look at what the cash flow projections would look like going forward,” says Ninah’s Murray.

Modelling of previous activity can help tell advertisers a number of things such as: “does it require a threshold of TV advertising; what’s a saturation level, so at what point does the advertising become inefficient,” says Griffiths

Econometricians need data to feed their statistical models, in particular good quality sales data, data on competitors and the market.

“You need to look at things like price and distribution, state of the market, state of the economy, not just the effects of your TV advertising but other channels as well,” says Binet. “Generally we are talking about wanting probably at least three years of data, at quite a detailed level and quite consistent, that’s just good marketing practice.”

“Most FMCG brand have good quality sales data, you do need to have a representative set of data for that,” adds Griffiths.

Traditionally it was a common complaint that agencies were not trusted with such sensitive data, however, that’s changed.

“Any marketing director interested in measure the effectiveness of what he does is wiling to let an analyst get access to the data,” says Binet, adding that “one area that can be sensitive is profit margins but models can be constructed so that clients playback these figures to themselves.”

The problem for some brands, particularly manufacturers who do not control their final consumer distribution, however, is knowing more than simply when the box left the factory.

“Quite often a client might have sold through [a retailer] and would not know at the end when a consumer has bought their product,” says Griffiths. In such cases you need to “find a few people you can source data from, as long as you know that this is robust or are at leased biased in a way that you understand, then you can have validity on your modelling”.

Time frame

Another issue for Murray is the time frame of the analysis. Restrict yourself to too short a period and you won’t see the full benefit of your TV work.

“We look at three-year impact not just when a campaign runs or a campaign year. Some FMCG products if you just look at your short term you’re not getting your pound back,” she says.

That’s a point echoed by Thinkbox’s Brennan. “Much of the drive to shift budgets online is driven by a very short-term analysis of sales results. While online may provide superficial accountability and hard numbers, looking at return on investment on anything other than a long term basis distorts or ignores the power of media like TV,” he says.

And that gets marketing into a whole new debate, the difference between accountability and return on investment. The two terms are often used interchangeably but actually there are significant differences. True accountability covers both a longer time scale and a wider set of measures where as ROI is more closely tied to sales success.

Thus it’s no surprise that there’s a debate about how much econometrics can tell you about longer-term effects of advertising. Carat’s Greenstreet argues that the impact that advertising has on consideration some way down the road is not recognised by such methods.

“For advertisers who believe that their TV advertising is about something more than just short-term sales return then econometrics isn’t the whole approach,” he says. “For many advertisers the role for advertising is to generate short-term sales but also to build the relationship their consumer has with the brand beyond that week or next month’s sales.”

MediaCom’s Griffiths also accepts that there are limits to the technique. “Econometrics tends to pick up the short term up-lift, we are talking a couple of months. Advertising has a much longer impact. It would also be naïve to just model your short term uplift by a factor of three,” he says. “Longer term benefits can vary enormously.”

Binet however, is firmly convinced that econometrics can tell you everything you need to know. “That line gets trotted out a lot and I think it’s over played,” he says, citing a paper he wrote as far back as 1990 on the impact of advertising on PG Tips over 20 years. “You can use measure long term effects with econometrics if you use the right techniques.”

Greenstreet adds that there are other tools of identifying the return on investment generated by TV adverting

“There’s a variety of approaches looking at trying to provide some conversion between a change in the way consumers think about a brand into their likelihood to purchase in the future.”

It’s particularly important he argues in areas where the purchase cycle is long, such as for mortgages or cars where the econometrics can be a weaker tool.

Carat Insight has developed a tool called Integrated Communications Evaluation to try to put hard numbers on which ads actually influence consumer attitudes.

“We ask them about their attitudes to the brand at the beginning, including questions about their experience of the product and then show them all the different executions,” he says. “We have a lot of information that we then use a modelling approach to determine exposure to which medium has had to various influences on the attitude to the brand.”

Where’s the cutting edge?

The technique enables brands to work out which attitudes are the strongest drivers of purchase intent and hence sales. He argues that it gives a better picture of the medium and longer-term impact of TV advertising.

“What the industry is now battling with is balancing short term considerations and long term consideration,” he says.

For Griffiths, the real issue of return on investment research is ensuring that the results of these studies are not just used to assess past work but also guide future activity.

“It’s about integrating the work back,” he says. “Less of a standalone ‘look aren’t we good, we’re doing ROI’ to making sure that all of that best practice and learning is taken through to all the different disciplines.”

That’s a verdict on which Murray agrees. The cutting edge of ROI is, for her, the use of on-going tracking and predictability, where “ROI tracking has become an ongoing metric in the business”.

She describes a major client in the cereals sector where ROI has become the key measure of success. “Every three months we update the information so they can see what their latest campaign has done, helping them to spot if for example the creative has worn out,” she says.

8 pointers to good evaluation

1. Set clear and ideally quantified objectives. Know exactly what return you are looking for.
2. Think about how you will evaluate your activity before it starts.
3. Allocate time and money to assess how the campaign performed went. One per cent of the budget is often cited as an appropriate figure.
4. Make sure you will be able to get the right data. Sales data measures what happened while awareness data helps you understand how it happened.
5. Make sure you have pre-campaign data that is appropriate. It will tell you what might have happened if you hadn’t advertised.
6. Ensure you take account of any other factors such as seasonality, changes in distribution or rival activity.
7. Look for relationships between your TV advertising and other communications activity. Is DM performance better when you on TV, for example.
8. But don’t let the evaluation tail control your media strategy.

Source: Evaluation, a best practice guide to evaluating the effects of your campaign.

Article by Alastair Ray