How to Explain Lies, Damn Lies, and Statistics

Don’t get me wrong: I like research. Survey information is grand. I just say don’t bet the store on it. Use it to educate your guesses, but only as long as you stay skeptical. Read it, consider it, but don’t believe it.

Mark Twain said:

There are lies, damn lies, and statistics

Blogger and business researcher Steve King, a sometimes-Twain-like research analyst (well, a smart person who lives in California, at least) gives a great example in his recent post Why Surveys Show Wide Differences in Small Business Social Media Use on Small Business Labs. Steve is a researcher, one of the founders of Emergent Research, which does a lot of work for Intuit.

Steve pulls up two surveys with starkly different results. The Wall Street Journal recently reported that 70% of small business owners think social media is important.  But a Citibank survey said only 36% of small businesses use social media and a mere 24% have found social media useful for finding leads or generating revenue.

What’s up with that? Methodology and sampling techniques, Steve explains. The survey that was big on social media was taken from people it found online using Twitter, Facebook, and other social media. The other one was a telephone survey.

So, as they say, “no duh.” Most of the business people who use social media think it’s important. Most of the ones caught on the phone don’t.

The point is that both surveys are valid in their specific context, both were done professionally, and both can help you understand what a defined group of people thought – or told survey takers they thought. But they contradict each other. So if you’re using research, use it well, explore the assumptions, look for the built-in slants, and take all of that into account.

Humans make decisions. Statistics don’t.


  • Stop. Don’t Do That Survey. Talk to People Instead. says:

    […] if you overuse it. Most surveys are seriously flawed. I’ve posted my problems with surveys here and here, but for today, you could read the lingering question of the survey revolution for a […]

  • Berislav Lopac says:

    I like the one which states that statistics is like lingerie: what it reveals is suggestive, but what it hides is essential. 🙂

  • Annie says:

    That is one of my favorite Mark Twain quotes. You have to analyze what you read carefully because people can pull just about anything out of surveys or polls to buttress their position. The simple fact is that social media does play a role in marketing today but it is not the only role. Good product and need cannot be minimized. Top analysts that follow the market often derive different opinions from the same research and companies pay folks for that. You have to read their reports and accept what seems to be the most logical.

  • Steve says:

    Tim: I’m a huge Mark Twain fan and have read almost everything he has written. Right now I’m trying to wade through volume 1 of his autobiography. Even for a fan, it’s tough sledding at 760 dense pages.

    Living in California and investing in tech startups that failed is about all I have in common with Twain. But since I consider him America’s greatest author and humorist, any comparison with him is a welcome one:).

Leave a Reply

Your email address will not be published. Required fields are marked *