Turns out the commenters over at Spin Sucks who didn’t know how to define ROI as based solely on dollars are not far off from many a marketer, according to a study by the Columbia Business School Center on Global Brand Leadership and the New York American Marketing Association.
By way of background, my friend Jenn Whinnem who is a contributor here and member of this community wrote a guest piece for Spin Sucks about the Connecticut Health Foundation and which metrics they tally that contribute to campaign success.
A major side conversation in her post’s comments ensued about whether metrics not attributable directly to dollars should be referenced as ROI at all. About 85 percent of commenters favored ROI as strictly dollar oriented while a portion of those who spoke up wanted to honestly know why ROI couldn’t be oriented to other types of measurement, as well.
In the March 12, 2012 Advertising Age, the cover story, Marketers Don’t Practice ROI They Preach, informs about results of a study that suggests marketers aren’t as buttoned up about ROI as the commenters in the Spin Sucks community would have you feel.
The major outcome of the study taken directly from the Ad Age article is this:
“For all marketing’s obsession with ROI, it’s not used to set budgets.”
and
“The state of the art remains surprisingly primitive.”
Interesting, eh? The discussion at Spin Sucks was heated yet collegial. Had I had this article tapped, I could’ve pulled it in to the conversation in comments (or, because ROI wasn’t on my radar until now, I wouldn’t have!). Let me share a few more bits of data from the survey as detailed in the magazine:
- 22% of respondents use the most basic measure — brand awareness — to gauge marketing ROI without necessarily determining even whether the awareness is positive.
- 50% of respondents didn’t include any financial outcome when defining marketing ROI.
Brighter Light Bulb
Don Sexton, marketing professor at Columbia Business School, stated in the article that “a lot of people…don’t have a clear idea of what marketing ROI is. A lot of them use metrics that don’t measure finance at all.”
These other metrics used to measure ROI by more than 1/5 of marketers surveyed (243), according to the Ad Age article are:
- Brand awareness
- Boss’s satisfaction
- Reach and frequency (regardless of whether it affected sales)
(Social media data (in spite of mass quantities) isn’t being used to measure financial metrics said the survey.)
Conclusion
The pulse of the Spin Sucks community on the definition of ROI seems to represent what this article and the survey results speak to.
- There is absolute confusion about how to implement marketing ROI and align it back to dollars.
- There is absolute confusion about the definition of return on investment.
- Somewhere along the way, use of the term, “ROI,” loosened to include metrics not associated with dollars.
ROI of a Webinar
Let’s take a look at a mini-campaign — a webinar to generate sales. In between the start and the end point are customer touches that influence sales and hopefully generate a purchase at some point in the sales cycle.
- Webinar invitation list to prospects and current customers
- Follow-up post event with a survey
- E-mail marketing push with the on-demand webinar
- Content to further interest the prospect in a product
- Sales team queries to webinar attendees
- Product demonstrations completed
- Customer buys the product (at some point)
There are many opportunities in this scenario to track metrics that help influence sales:
- Number registered for webinar
- Number that attended
- Number of completed post-event surveys
- Percentage of positive survey responses
- Number of product demos scheduled
But, the only true assessment of ROI for this webinar is whether a customer bought a product — that is pure and true ROI. The total number of dollars exchanged to make a purchase is what defines ROI. The other numbers, as suggested above, are metrics that contribute to the success of the campaign and help measure its effectiveness or influence on sales.
What is your opinion about ROI and how it’s used or defined in your organization?
Since I work in government, ROI is not something that used in the traditional sense of the term. We don’t have a product to sell; we have a service to provide. So I really can’t use that measurement tool in the traditional and purest sense. But measurement is important. A local social media consultancy here in Columbus likes to refer to the ‘I’ as ‘influence.’ And I get that. It’s hard to but a financial value on Likes, Shares, Visitors, etc. And if we did that, I feel it would have the same flawed result as advertising equivalency. None of this guarantees sales but it does help influence. And there is value to be placed on an organization being able to capture the interest of people to the degree that they keep coming back or share the message with others.
Sorry. It’s hard to *put* a financial value on Likes, Shares, Visitors, etc.
@Anthony_Rodriguez What you say here is so valuable to the broader discussion about how we prove results and outcomes. Do we say ROI (loosely or oriented to the $ signs it defines) or do we say metrics ala yours above and jennwhinnem in her not-for-profit/foundation.
For me, the term is used so loosely that when a pure marketer accustomed to pulling dollars into conversions is in the mix, there is definite confusion over how.
Measurement, and I hope that Shonali Burke has a moment to stop in here, offers another critical view on campaign success, metrics, results, ROI and proof points.
I don’t think we’re all saying the same thing, either! Great having you today, Anthony. Thanks.
I can’t define it, but I know what it looks like when I see it……………:).
@bdorman264 Uhmm, I think it’s all about giving peeps +K; that’s ROI, right?
I just did a group presentation on social media ROI for my social media class and I’m STILL confused about what exactly it is. Even after talking to some professionals we couldn’t get a real solid answer. I kind of feel like social media is so influence based and as much as that does help sales we don’t really have an easy way of proving or tracking that.
And side note, this: “22% of respondents use the most basic measure — brand awareness — to gauge marketing ROI without necessarily determining even whether the awareness is positive.” BAFFLES me! How do you not even track whether or not the discussion about your company is positive or not! That’s some lazy marketing!
@asdupre Heh. And, I didn’t even use the best quote from the Ad Age story with some respondent who said “Calculating ROI was like pissing in the wind.”
One conclusive bit of info…social media data aren’t being used to calculate much yet…this is an unfolding quagmire and anyone who manages to harness it will be training the rest of us for sure.
Thanks for stopping in, Lady.
@asdupre Could part of the “not tracking pos/neg sentiment” be that it’s really hard to do, never mind automate that? I’ve been told that even Radian6 is only about 60% accurate.
@jennwhinnem I’ve never used Radian6 and maybe I’m being a bit naive since I’m still only a student but it can’t be that much more difficult that monitoring the conversations occurring. Time consuming, yes but I don’t think difficult. Also, I’ve used socialmention.com a little bit and it calculates pos/neg/neutral mentions of the keyword you search. Very interesting tool, check it out!
@jennwhinnem @asdupre I still have so much to learn in that regard. Listening tools and analytics reports deliver a treasure of data, and this is what I hear — you need an accounting degree to interpret.
I need to think about this. Mention ROI and people show up with their knives and guns. I think there’s a deep insecurity on the part of people in social media and it plays out in nastiness.
Here’s what I do know. You always have to show that you’re having a positive impact on the bottom line. I agree with that. Otherwise I don’t have much to say, but I appreciate you shining an illuminating beam on this issue friend Jayme.
I sense a balloon without air. The blogosphere can be a highly aggressive place; not for the faint of heart (did that come from ancient England?). That said, it’s all a learning experience; each of us can recall a time when we got blasted and it was painful. Time goes by, and we reference it as a learning experience and teachable thing for others who’ve yet to experience that horror. @jennwhinnem
For you, meine Freundin/mi amiga, your writing is so above quality that it would be a sad day to not see you contributing on a more frequent basis. When you leave the channels you once frequented and try to come back once every six months or more, you forget how it’s like.
So, consider this your formal (you don’t need one) invitation…I’d like to see you write about this new woman power thing I’m reading about…have you seen it?
@Soulati | B2B Social Media Marketing No what are you talking about?? I want to read it!
@jennwhinnem @Soulati | B2B Social Media Marketing I am always kind and gentle with my blog. Wait til Subway sees my opinion on their latest Battleship game. Warm and fuzzy even coddlingly supportive.
Oh, man. That’s #RockHot — “coddlingly.” #thatIsall @HowieSPM @jennwhinnem
Some story in Wall Street Journal last week I clipped out. Apparently, women are really pulling it together and getting awfully serious about life as independents or some such. I’ll try to find it for you!
I guess Project Eve is tres timely! @jennwhinnem
This is a GREAT POST @Soulati | B2B Social Media Marketing
Why do marketers not like set budgets? so they can inflate what they spend like the Government. I bring this up often how Agencies goals (billings) and Clients goals (more sales per marketing dollar spent) are at odds. Marketers like cost plus projects. And they hide behind their view that they can’t measure. When I was trying to launch my Skrib Sheetz product I would call on Media Buying Agencies like Starcom. They get 15% commission on every dollar they spend for their clients. See what is wrong with the incentive?
Now I told Jenn roi doesn’t have to be sales oriented but you can financially figure it out. The problem marketers have is the CFO. His job is to allocate funds for the highest return possible for either the owners or stockholders. They will use Net Present Value – @bdorman264 knows this stuff. and figure out if they have X $$’s to spend which gives the highest return. So if Manufacturing says buy a new machine save X $$s and marketing says give me the same money buy I am clueless if it will make money the CFO gives the money to manufacturing. This doesn’t mean it is the right choice! But accountants and money folks like concrete numbers. They don’t like ‘Branding, Engagement, Impressions etc’ they like input X dollars i get X dollars back.
Marketing for 50 years went on the warm and fuzzy. Now there is pressure to give ROI proof in some sense and marketers tend to push back because it hurts their cash cow. On this front Starcom has been forced by some mega brands like Coca Cola to prove a ROI of they have to rebate dollars. The CFO of Coca Cola gets bonus based on share price and if Sales says I can make you X money but marketing says ‘I think I can make you X money’ you know where the CFO is allocating funds. Bird in the hand….
@HowieSPM @Soulati | B2B Social Media Marketing @bdorman264 Am in complete agreement with paragraph 2. I have had the CFO ask what the ROI of social media for us is (and now she contributes some of our most-read content!). The money to justify, in my case, would be my salary + benefits, as well as the cost of the new website that I helped launch once I started. That’s the investment. As for the return, that is, as niecolec discussed in the comments, greater awareness. That’s what we want.
@jennwhinnem @HowieSPM @bdorman264 niecolec That you’re VERY clear as an organization how you measure value is what matters most, Jenn. We all know, what works for some doesn’t for others yet the book definition provides the foundation to start us out of the gate. (That would be ROI, of course.)
@HowieSPM @bdorman264 Thank you, Howie! And, so appreciate you came. Your mind; I LOVE your mind. #ThatIsAll 2
Well. I think it’s a great post. ROI is the most important thing that CFOs want. Now, What can we do? Well. I work in an ageny and one of our clients is a clothing company. They need to sell. Period. They’re a little bit afraid about social media impact and sale results. They trust us. One thing is that we can measure a single campaign and see the results of it, but another thing is to know how many people actually went to the store and bought something as a result of the campaign.
How can we know that? Should we ask when he or she is paying in the store? Just asking.
@Tito Bohorquez Gaitan Hi, Tito; great seeing you here! Also, a good question. For every campaign there are ways to measure success. For retail sales, did you see peak volume when you launched a campaign, when TV spots ran, when a celebrity or other influencer said to go buy that product?
The timing for an increase in retail sales volume should align with your marketing efforts. The impact of Groupon on sales can be directly attributable to higher volume; however, ROI is likely to suffer because retailers often lose money with these deals.
I think there is a difference between ROI and results. ROI is return on investment, which translates to money. Time also is money so, even if you’re using social media tools that are free (no money invested), they take an inordinate amount of time so there is a return there.
The problem with brand awareness is it isn’t measurable. We know if we have it. It feels good when we have it. It’s easier to sell if we have it, but you can’t tie it to the time and money spent. You can, however, track it in results.
The fact that 50% of the respondents don’t use financial outcome when defining marketing ROI doesn’t mean you don’t have to do it. It means half of the industry doesn’t know how. As a business owner, this makes me nuts. It’s now easier to measure a return on your investment, yet half of the respondents don’t do it? That’s insane.
As for the Spin Sucks blog post…Jenn absolutely wrote the right post, but it should have said “results,” not ROI. That’s our fault, not hers, for changing the headline.
@ginidietrich It was eye opening to see the comments at your house and then to tap into that article as above. I had no idea people were unsure of how to define ROI and how broadly the term is used.
@Soulati | B2B Social Media Marketing @ginidietrich @jennwhinnem Jenn’s piece was definitely eye-opening in that regard. It seems ROI has become a general buzzword to define results. It all comes down to goals and what you’re trying to accomplish…that dictates what you measure. Can’t wait to see you three in two days!!! 🙂
@lauraclick @ginidietrich @jennwhinnem The schools of thought and actual practice on this are vastly different; it’s now obvious. Thanks, Laura. See you Thursday!
@ginidietrich I think Gini, addressed what I was getting at, in a way.
Thank ya 🙂
One thing though, saying you *cant* tied brand awareness to ROI feels like a cop-out to me.
It might be something humanity isn’t able to measure… yet… but it doesn’t mean they’re not related and in fact, it seems FAR more likely that they ARE related.
Simple pattern extrapolation can tell us that companies with significantly higher brand awareness consistently, for hundreds of years or more, have enjoyed higher ROI. Period.
Is it really a stretch to say they’re tied together, whether measurable or not?
@Jason Fonceca @ginidietrich Gini may have a more solid response for you. In my view as yours, you’re right it’s all marketing and they’re related. But, the purists will never let anyone think that ROI is anything but defined as dollars; done.
@Soulati | B2B Social Media Marketing @Jason Fonceca I agree they are related, but not when talking about dollars and cents. Unfortunately, when you’re talking to a CEO or CFO, saying humanity isn’t able to measure it yet, but we know it’s good doesn’t cut it. That’s why it’s so important to have a completely integrated program so you can have the stuff the executive team cares about combined with the stuff we intrinsically know grows a business.
@ginidietrich @Soulati | B2B Social Media Marketing Gini, I definitely see what you’re getting at, and I appreciate having an expert perspective.
I have no ‘dog in the fight’ as it were, but I do love bringing fresh ideas to discussions 🙂
I particularly like your integrated program approach. It even sounds smooth 🙂
And I agree that “its good” doesn’t work for the vast majority of people — but I also say that the world of business isn’t as driven by “all powerful numbers” as much as people think.
It’s founded on dreams and gut instinct a lot of the time, and successfully so, and II make allowances for that.
For example, it might depend on which “CEO” you’re talking to:
I’m willing to bet Richard Branson has made decisions on less than that, and Buffet’s famous for doing billionaire dollar deals, that most companies would take years to analyze, and cementing those deals with minimal due diligence, trust, and a hand-shake.
Food for thought anyway 🙂 Fantastic post and discussion, so glad to be here.
@Jason Fonceca @ginidietrich So glad you’re here, too, Jason! You do bring a valued perspective and insightful approach. You prompt others to think harder; that’s #RockHot. Thank you!
I’m curious… if a big campaign makes very little sales, but gets the attention of saaaay Warren Buffet or Kim Kardashian and they jump on board and lend their dollars to the cause/brand — is that ROI?
Because I don’t think you’d convince whoever ran the campaign that it wasn’t 🙂
@Jason Fonceca Hi, Jason. What you’re saying at the top of the comment is brand awareness. When you say they “lend” their dollars, you mean they donate, right?
In my opinion, that’s something entirely different than ROI; however, each organization needs to define a campaign’s success metrics. If fund-raising from influencers was a communication strategy and it occurred, you met your goal.
Did you spend money courting those donors; how much? Did donations outweigh out-of-pocket expenses to put on the event/campaign to fund-raise?
I’d say in a non-profit, people would be inclined to say that’s ROI.
@Soulati | B2B Social Media Marketing Yeah, I meant donate/invest 🙂
I was more getting at “giant results” that were NOT specifically intended/measured metrics.
Because if that happens to someone, you can’t really go to that CEO and say “your campaign sucked, it totally missed the mark on every metric”, and the CEO’s like “yeah, but now I’m friends with Buffet — who’re you?”
@Jason Fonceca I see your point. No kidding; anyone who happens to open a flood gate with results like that is golden. Who cares how anyone defines ROI at that point; the fact is…$$ just rolled in.
I’m not an analytics marketer nor a CFO or accountant. Numbers I use, but they don’t define everything I do. You can bet that donations into coffers would be a huge measurement for me. Then, when we calculated $$ against expenses and learned we were in the black; that’s ROI.
@Jason Fonceca @Soulati | B2B Social Media Marketing Ha ha! That is great!
Thanks Jason for some different thinking on this topic. Jayme the facilitator!
@jennwhinnem @Soulati | B2B Social Media Marketing Thanks so much Jenn and Jayme — I’m famous for fresh views on taboos (which is the name of my upcoming video series :P)
Well, it’s just carrying the “results speak volumes” idea to a bit of an extreme.
It also touches on “follow your heart”, as applied to campaigns.
It does fly in the face of traditional ROI, but that’s not to de-value it.
I treasure people who can help me feel clear and solid on the results I’ll be reaping 🙂 Thank you analysts, PR peeps, CFOs, and more 😉 I love ya.
@Jason Fonceca @jennwhinnem Shoot; at least we PR peeps are good for something as regarded by someone! Thanks, Jason!
Remember the old marketing saying… “I know 50% of my marketing is working, I just don’t know which 50%.” This is an age old problem in marketing because the direct correlation between investment and return is often hard to find. Some things are easy to track — your webinar example for instance — others, like a large multi-channel campaign can be tougher.
Technology has given better ways to establish linkage — dedicated URLs, 800 #s, etc. — and even with those, it can be hard for marketers to show that even if a campaign had a positive ROI, which pieces of the campaign were responsible and to what degree.
As your article said: ”The state of the art remains surprisingly primitive.”
@adamtoporek What’s fascinating the most to me is the flak PR peeps take for not adopting measurement, business analytics and more marketing strategy/tactics. Now, here we are realizing that marketers have their own issues defining ROI and measuring campaign success. How ironic.
Love the post, Jayme. And I wanted to note to you and your readers that you can download the full (free) study at: https://j.mp/MarketingROIstudy. We are continuing our analysis of the data in other ways, so stay tuned for updates….
There is a lot of great discussion here, and I think two excellent points are raised:
1. When conducting a marketing activity, defining clear goals before undertaking the effort and then then finding a metric to determine how well they have been met is key.
2. It is important for marketers to clearly understand ROI, and to work towards connecting their marketing efforts to ROI. As noted, to determine a return on investment one has to measure first the costs (financial & time) that were put into the effort and compare that against what the financial gains returned.
The important thing is to begin creating measurement methods and metrics to track PR and marketing efforts, and to begin developing links to financial returns. Even if at the start you can find only correlations between marketing/PR campaigns in cases where direct causation can’t be fully attributed.
Regards,
Matt
Matt QuintCenter on Global Brand Leadership
Columbia Business SchoolProducer of the BRITE Conference
Just had the folks who did the study as shared above in Ad Age send me link to the free study via Twitter.
Shoot, LiveFyre doesn’t allow urls to paste in comments? Here we go:
www4.gsb.columbia.edu/globalbrands/research/brite-nyama-study