How to Explain Lies, Damn Lies, and Statistics

Don’t get me wrong: I like research. Survey information is grand. I just say don’t bet the store on it. Use it to educate your guesses, but only as long as you stay skeptical. Read it, consider it, but don’t believe it.

Mark Twain said:

There are lies, damn lies, and statistics

Blogger and business researcher Steve King, a sometimes-Twain-like research analyst (well, a smart person who lives in California, at least) gives a great example in his recent post Why Surveys Show Wide Differences in Small Business Social Media Use on Small Business Labs. Steve is a researcher, one of the founders of Emergent Research, which does a lot of work for Intuit.

Steve pulls up two surveys with starkly different results. The Wall Street Journal recently reported that 70% of small business owners think social media is important.  But a Citibank survey said only 36% of small businesses use social media and a mere 24% have found social media useful for finding leads or generating revenue.

What’s up with that? Methodology and sampling techniques, Steve explains. The survey that was big on social media was taken from people it found online using Twitter, Facebook, and other social media. The other one was a telephone survey.

So, as they say, “no duh.” Most of the business people who use social media think it’s important. Most of the ones caught on the phone don’t.

The point is that both surveys are valid in their specific context, both were done professionally, and both can help you understand what a defined group of people thought – or told survey takers they thought. But they contradict each other. So if you’re using research, use it well, explore the assumptions, look for the built-in slants, and take all of that into account.

Humans make decisions. Statistics don’t.