Monday, May 9, 2011

Unilever’s Dove uses dubious “Go Sleeveless” poll to address criticism

This post originally appeared on my old “Joe Dellosa on Advertising” blog.

► Dove, criticized for inventing an insecurity for women, is justifying its new Go Sleeveless Deodorant by pointing to a study that says 93 percent of women think their underarms are unattractive but is refusing to release the actual poll or the full results.

This is an introduction of sorts to a fuller piece about Dove, its marketing campaign for the Dove Go Sleeveless Deodorant, and the inconsistency of its "pro-self esteem" brand image. The expanded piece will appear either later today or tomorrow.

Last month, Unilever's Dove began a marketing campaign to promote its new Dove Go Sleeveless Deodorant. The TV commercials for the deodorant promised women that they could have more beautiful underarms after using the product for just five days, which prompted many in the media to accuse Dove of attempting to invent a fabricated insecurity for women -- uh-oh, unattractive underarms! -- and profit from the resulting anxiety. Here's Stephen Colbert on the April 13 episode of The Colbert Report.



That a company in the beauty industry is trying to foment negative feelings in women about their bodies is certainly no surprise, but for Dove to do it seems hypocritical. Dove, after all, actively cultivates a reputation as a supportive, empowering brand for young women -- from its Campaign for Real Beauty to its Movement for Self-Esteem.

Possibly to fend off accusations of hypocrisy, Dove emphasized repeatedly in its press release about Dove Go Sleeveless Deodorant that the deodorant is the result of market research that found that "almost all women feel their underarms are unattractive" -- 93 percent, apparently. (This statistic was repeated unquestioningly in media outlets like the Wall Street Journal and WNBC.) One can imagine Dove arguing, poll in hand, that it's not inventing a new insecurity but merely addressing a pre-existing insecurity.

Unfortunately, the press release contained very little information about the poll itself -- no mention of the exact questions asked, how the poll was conducted, or any discussion of the methodology. It's even not clear if the poll is really statistically valid.

So, I asked Dove if I could take a look at the poll myself. I wanted to see the complete questionnaire, with the exact wording and exact order of the questions, and I wanted to see the complete results. I also had specific questions about the poll's methodology, including whether a random sample was used, how participants were recruited, and what the margin of error for the poll was. After a few emails back and forth, Dove, through its PR firm, declined to let me see the complete questionnaire and results and chose not to answer my questions about methodology. Apparently, I was to take Dove's word for it.

That, needless to say, is a bad idea. If a company is willing to mention some poll results -- presumably, the results that benefit the company -- in its press releases and other marketing materials, it should be willing to release the full results, with a full questionnaire, with a full explanation of the poll's methodology. To do any less isn't just shady and dishonest; it also insults the intelligence of that company's customers and undermine that company's credibility. More importantly, reporters -- especially those who cover advertising and media -- shouldn't let companies get away with it.

What follows below is kind of a Stats 101 mini-primer as to why seeing the questionnaire -- with the exact wording, exact sequence of questions, and full results -- is so important, and why, without it, even a statistically-valid poll can be completely dishonest and manipulative. Since Dove refuses to release any of that, I can only speculate as to how it got the results it did. For all I know, Dove asked fair questions that yielded legitimate results; however, until Dove decides to be more forthcoming, we'd do well to assume the worst.

 Exact wording

Consider the most prominent statistic in the marketing campaign, the “93 percent of women think their underarms are unattractive” stat. It could be Dove just very plainly asked:

Do you think your underarms are unattractive?

A: Yes
B: No

Would 496 women out of 534 women in a supposedly representative sample say they have an active belief that their underarms are unattractive? That seems unlikely. So what if Dove actually asked this:

Do you think your underarms are attractive?

A: Yes
B: No

Only 38 women answering "yes" still seems low, but it’s a little more plausible than 496 actively affirming a belief that their underarms are unattractive. And in this case, if 496 women answered “no,” they’re not necessarily saying they think their underarms are unattractive; they just never thought of describing their underarms as particularly attractive. (It's like if someone asked me if I thought my index finger is attractive -- I'd answer "no," not because I think it's ugly, but because, as a normal index finger, it's neither attractive or unattractive.) Of course, without the original question phrasing, we just don't know.

 Full results

Now, what if the Dove poll asked the question in this manner (with hypothetical results in brackets):

How do you feel about your underarms’ appearance?

A: My underarms are always attractive. [7%]
B: Every once in a while, my underarms don’t look the best. [50%]
C: Occasionally, my underarms don’t look the best. [30%]
D: Often, my underarms don’t look the best. [10%]
E: My underarms are always unattractive. [3%]

I could imagine that, presented with these choices, only 7 percent of respondents affirming that their underarms are always attractive, with a vast majority picking less absolute choices. But if Dove wanted the statistic that 93 percent of women think their underarms are unattractive, it can just lump all the non-“always attractive” responses together -- it's a tricky interpretation of data that could be seen as technically true but is patently dishonest. Hence, the importance of full results.

 Exact sequence of questions

Finally, consider if before asking about their underarms’ appearance, respondents were asked a series of questions like these:

Have you ever noticed any bumps or pimples on your underarms?

Have you ever noticed how rough your underarms can be?

Have you ever noticed any underarm decoloration?

Do you ever experience soreness or itchiness on your underarms?

Can you recall any times when you’ve been embarrassed by your underarms?

Followed up with an innocent, “Do you think your underarms are unattractive?” Clearly, the responses would be more than a little skewed.

It’s like asking, “Do you think Senator Jones is lying about having an affair?,” “Do you agree with Senator Jones’s decision to defund the troops, possibly leaving them without body armor?,” and “Do you think Senator Jones’s health care vote was the result of bribery, as some have suggested?”—and then asking, “Do you approve or disapprove of the job Senator Jones is doing?”

 In closing (for now)

I'll have much more on this in a fuller blog post about Dove, including an explanation of the inherent inconsistency of Dove's Campaign for Real Beauty and why I believe its campaign for its Go Sleeveless Deodorant is embarrassingly hypocritical. (I'm holding off posting that in part to give time for Dove to provide a statement and, hopefully, the full poll; a rep at Dove's PR firm said they'd have something by today.)

In the meantime, it's important to note that Dove is far from the only company to conduct dubious polls, cite the results in its marketing materials, and then balk that the notion of releasing the full questionnaire and results.

For instance, last year, FedEx attempted to smear UPS with its "Brown Bailout" campaign, in which FedEx falsely accused UPS of asking for a government bailout. FedEx cited polls on its website and in its press releases claiming public support for FedEx's position, and FedEx director of corporate communication Maury Lane mentioned polls when he was interviewed about the issue. (He did so during his interview with me, too.)

So I asked to see the full questionnaire and results. Lane told me that I would have to get it from FedEx's pollster, Public Opinion Strategies. Public Opinion Strategies said that they weren't authorized to release that information, and only FedEx could release it. When I emailed and left messages with Lane relaying what the pollster told me, Lane stopped returning my calls and emails.

This may be obvious, but it's a good idea to be suspicious of any poll a company conducts and then cites in marketing materials. Companies don't do polls because they're just so darn curious about people; they do polls because it helps them somehow -- maybe to have snappy soundbites in interviews, or to make press releases sound more authoritative, or, in what I believe is Dove's case, to attempt to preemptively fend off criticism.

Regardless, whenever companies offer poll results in a marketing campaign, customers and (especially) journalists should get into the habit of automatically asking for the full questionnaire and the full results. And if a company refuses to do so, we should get into the habit of telling them that they are being dishonest and very likely lying by omission.

There's only one reason a company would feel comfortable releasing only some but not all of a poll's results: they are hiding something. That's not breaking news, but it's good to remember.

You can email me at jdellosa@gmail.com, or tweet me @JoeDellosa.

No comments:

Post a Comment