I can’t resist pulling a couple of posts together today to follow up on truth, lies, statistics, research and, ultimately, metrics in business. We should at least recognize, if we’re going to be in business and we care about ethics, that there are ranges and degrees of truth. It isn’t black and white.
We have to manage that problem of truth on several levels. First, there’s the example that
Mark Hurst posted as More on wine and charlatans over at Good Experience Blog today:
The experiment, conducted by Cornell professors Brian Wansink and Collin Payne, was fairly simple. A group of diners in an Illinois restaurant were served a free glass of Cabernet Sauvignon with the same fixed-price French dinner. All were told the wine came from Noah’s Winery (a non-existent brand), but half were told the origin was California and the other half North Dakota. In fact, all of the wine was “Two Buck Chuck,” a very inexpensive but brisk-selling wine from Charles Shaw Wines. Predictably, the “California” wine was rated as being better than the “North Dakota” wine.
Here’s the interesting part, though: the diners who received the free glass of California wine also rated the food higher, ate 11% more food, and were more likely to make a return reservation.
So this story illustrates levels of truth, and ways to manipulate. Presumably we all agree that the restaurant can’t lie about its wines. But are the people who liked the food better and ate more also lying? And what does this tell us about customer surveys? When a seemingly unrelated factor creates this much distortion in the data, how much to we want to use this to make decisions?
What if a restaurant is aware of a survey and decides to give free California wine that night? Is that dishonest?
We like to distinguish quantitative data from qualitative data and most of us trust the quantitative more. If you read that people liked the food better, that’s not nearly as convincing as a number, in this case 11% more food. But how easy is it to manipulate that number?
And that brings me to Marc Andreessen’s post from this morning also, titled You have to love this industry, in which he tracks a self-serving switch by Microsoft CEO Steve Ballmer:
Oct. 2:
Steve Ballmer, the Microsoft chief executive, believes that the craze for individual social networks such as Facebook risks being exposed as a “fad”…
“I think these things [social networks] are going to have some legs, and yet there’s a faddishness, a faddish nature about anything that basically appeals to younger people,” Mr Ballmer told Times Online yesterday…
[Ballmer] added that there was little in the way of technology to justify the lofted valuation attached to a site expected to achieve revenues of only $150 million this year.
“There can’t be any more deep technology in Facebook than what dozens of people could write in a couple of years. That’s for sure,” he said.
Oct. 24
Microsoft Corp. agreed to invest $240 million for a 1.6% stake in Facebook Inc. that values the social-networking site at $15 billion, beating Google Inc. in a closely watched contest.
As part of the deal, the two companies expanded their existing advertising agreement. Microsoft, which previously handled Facebook’s U.S. ad sales, will now also sell the site’s international advertising.
I think this is amusing, but not dishonest. Opinions are not fact and business people are supposed to manage opinions and even information. That’s his job. Actually, given our securities laws, executives of publicly traded companies can be liable for honesty when they are honestly optimistic about the future and then they turn out to be wrong. I think Steve is doing exactly what he’s supposed to do.
Put it on more personal terms: you’re interested in buying a certain house. As you walk through the house you make negative comments that the seller’s representative can hear: it doesn’t have enough light, the appliances are old, etc. Your intention is to influence price negotiations to come. Is that dishonest, or just good business?
Now put it into a different case. Whole Foods CEO John Mackey uses a fake name to comment on his company and a company it wants to buy on stock-related sites. Is that dishonest?
And now I can make it personal, and specific to me. I’ve been an employer for many years now, which means occasionally I’m asked to give a reference on somebody who no longer works for my company. I have several times failed to give a full and fair airing of negatives about that former employee. Why? Because I don’t have to and I don’t want to. Am I lying by omission?
How about you, when a survey taker asks your opinion about a product: do you say it’s too expensive just to manipulate the manufacturer to lower the price? When they ask you how much you intend to spend, do you increase the amount because it makes you feel more important? Do you assume honesty matters in a survey?
I don’t know about all this but I’m interested. I think the restaurant that lies about what it’s serving is dead wrong, and bad. I think the business person who shares negative opinions while negotiating a deal is doing the job. I think the business person who gives a falsely positive review of a former employee might be doing the job too. What do you think? Does it matter? Is there a right and wrong?
By 
Comments
This is a wonderful post that provides great stories about common types of bias in social research. In their book, "The Psychology of Survey Response," Tourangeau, Rips and Rasinski suggest that there are four areas of question processing that can shed light on the filters or biases that come into play when answering questions (excuse the broad paraphrasing):
1. Comprehension of the question: how clear is the question; socio-demographic biases
2. Retrieval: ways of remembering and accessing; specific and generic memories; filling in missing data
3. Judgment: decisions concerning information needed for an accurate answer; estimates based on partial or inaccurate memories
4. Response: evaluation of response in terms of accuracy; evaluate in terms of other goals.
With that in mind, to answer your question, one needs to understand the filters and biases of those asking the questions and those answering them and try to make sense of the responses. It also depends on the cost of making the decision based on poor information. In the case of Microsoft, perhaps the response saved the company millions in litigation and a higher valuation of Facebook. In the case of the employee referral, you could hurt your company / team moral if you only rely on testimonials and don't do other evaluation.
Again, thanks for the post.