Planning, Startups, Stories


Tim Berry on business planning, starting and growing your business, and having a life in the meantime.

Can Research Make You Dumber? 4

(Reposted with permission from my social media business plans blog)

Can research make you dumber? It can if you believe it.

I just read Can Facebook Make You Fat and Poor? on Mashable. It’s a post by David Mielach, of BusinessNewsDaily.

In particular, the researchers found that social media users were more likely to binge eat and have a higher body-mass index. Frequent Facebook users also were more likely to have certain financial problems, including a lower credit score and higher levels of debt.

But wait. It says the research was based on the responses of 541 Facebook users in the United States. So what does that really mean? What does this research really mean? And to be fair, I haven’t gone into the actual research. I’m just commenting on the coverage. Maybe they did everything right and avoided the problems I see. And maybe not.

First, who’s in the sample? Is it Facebook users, really, or Facebook users who answer surveys? Those are different sets of people. Is it balanced for age, demographic, technology, geography?

Maybe people who answer surveys have less self control, which is part of the reason they answer surveys. And maybe people who answer surveys have less money, caused perhaps by the behavior that finds time to answer surveys. Maybe they are just younger, on average, and that causes the money difference.

Research depends on the sample. So that’s a good reason to be skeptical.

So maybe what it really shows isn’t about Facebook users but rather about people who answer surveys. Maybe they — survey answerers have less self control so they couldn’t resist taking the survey really know is that people who answer surveys on Facebook have less self control — that’s why they took their time to answer the survey. And maybe people who answer surveys have less money — because they waste their time answering surveys.

And there is that whole issue of causation and correlation: Could we just as easily say living in a large house makes you rich, or attending college makes you young? That’s as logical as saying Facebook users have less self control and less money. Right?

Here’s a direct quote from the research:

These results are concerning given the increased time people spend using social networks, as well as the worldwide proliferation of access to social networks anywhere, anytime via smartphones and other gadgets. Given that self-control is important for maintaining social order and personal well-being, this subtle effect could have widespread impact.

So now it’s widespread impact. The emphasis above is mine. Wow: Is this looking for a news lead, or rather reaching out, stretching to the ultimate, to look for a news lead? Or what?

I’m not saying that information is bad. Misinformation is.

I’m not saying that research is bad. Believing it is. Question the research, question the assumptions, look through it, and then take what’s valuable in it. Never just believe it.

  • http://www.facebook.com/nicholasrobinsondesign Nicholas Robinson

    I agree with Tim and the group,
    Having used facebook too, suffered from overweight and incurred financial problems, I decided to go to the gym and take back some control a year ago, rather than become a ‘purveyor of the hod’ or such. I tend to use facebook more for blogging about business promotion and development rather than researching per se, to keep things positive. What kind of research do you carry out using facebook? I would be fascinated to know.

  • http://www.kerstenanderson.com Kersten Anderson

    I totally agree, even with what Clint said. My MBA days told me to look at the source and how the information was gathered before ever saying the information was accurate. Most people today do not care about accuracy, but about big flashy numbers and stats that sound good. I don’t care what stat you say, as long as you can back it up with how you obtained the information. If your stat comes from a voluntary online survey that is not random, then state that and stop trying to pass your information as representative of the population.

    Thanks for the post and reminder of my days of stats…at least some people still value accuracy and are not blinded by flashy numbers.

  • http://quemarketing.wordpress.com/ Emilio Munoz I Marketing On line

    Interesting reflection. I think there’s no correlation between the data because everyone uses social networks. Wait a minute, this crisis, all people in debt … what about social media? :D

  • http://www.cazoomi.com Clint Wilson

    Great TGIF read Tim and totally agree.

    From my MBA stats days I think you have to have a sample of 1500 to even be relevant +-3% and certainly this statement, “research was based on the responses of 541 Facebook users in the United States. “, would not even be relevant to our customers at our CnC Cafe Internet in the Philippines who are not fat or in debt for sure:)

    Data from any angle looks different.

    ~Clint
    @cazoomi