A CERTAIN POINT OF VIEW

starwars6_clip12[1]

“Statistics are the triumph of the quantitative method, and the quantitative method is the victory of sterility and death.” Hilaire Belloc

WARNING – A small rant here because 140 characters on Twitter was ineffectual for the purpose.

Last week, Personnel Today published an article headlined “A quarter of HR directors say their own HR function is ineffective”. What troubles me most about the article is not it’s central theme, although having spent the better part of the last six months listening to the discourse of a broad network of HR professionals, I sense an enormous amount of effectiveness along with a passionate desire for improvement.

No, what provoked my ire was the way in which data was used to support the assertion. The survey from which the data is taken was of 67 HR leaders from some of the UK’s biggest organisations, with a combined total of 1.1 million employees, in both public and private sectors.

Most of the useful data wasn’t provided so we’ll make some hefty assumptions of our own. As at June 2011 the constituents of the FTSE All-Share Index totalled 627 companies. We’ll assume all of the private sector leaders were in this group. Let’s assume that the public/private sector split was 50/50 for balance (33.5 persons). That means that 5.34% of the All-Share Index businesses are represented. A quarter of these folks (8.4 persons) thought their HR function was ineffective. That’s 1.34% of All-Share Index companies with a problem. And a markedly different headline:
“98.66% of FTSE All-Share Index Businesses Have Effective HR Functions”

There are three kinds of lies: Lies, damned lies and statistics

Data and metrics are incredibly important and powerful. They also provoke a great degree of scepticism, much of it justified. Scientific data and research is subjected to the most rigourous (and often rancourous) of peer review processes. That used for marketing or business propaganda rarely is (unless there is sufficient public interest). Articles such as the one in Personnel Today, do nothing to convert the sceptics and unhelpfully paint entire professions in a very poor light.

Right. Rant over. Want to be inspired by a story about statistics? Try “The Median Isn’t The Message“, by influential evolutionary biologist Stephen Jay Gould

I am indebted to David Goddin (@ChangeContinuum over on Twitter) for sharing the article that prompted this post and for tolerating the minor rant that it set off.

Advertisements

8 comments

  1. […] of the data population in this article published in Personnel Today. He’s since written this piece putting the statistical perspective on the […]

  2. Mark Eltringham · · Reply

    To be fair Simon, your own statistical conclusion is flawed. 🙂 What you need to determine is whether the survey has a statistically meaningful sample size – so you can’t say that 98.66 percent of HR functions are effective.

    Personally, I’d say 67 responses to a questionnaire would give you a ‘report’ rather than a ‘survey’ because you can’t draw conclusions from that and certainly not one as self-serving as this. In the case of this survey, I actually think what they’ve done is counterproductive because if I was a HR Director, I’d think they were treating me like a fucking idiot.

    On the general point, I work on the basis that people are able to make allowances for the commercial nature of such stuff. Today we published a the results of a survey about social media use by Microsoft, owner of social media business Yammer. Obviously they ask leading questions and don’t publish detailed information about their data gathering so I think we should trust people to season the results themselves.

    1. You’re right if course Mark and that was part of the point I’m making. I hope my caveats about assumptions and warnings of a rant were sufficient for less clued up readers than yourself.

  3. Data is an invaluable part of our toolkit, but the problem with it is that it is so damn appealing. People have been twisting words for far longer than statistics, but we are used to people telling lies – it still feels odd when a statistic turns out to have either been made up or turns out to be misleading.

    You can only trust statistics as far as you trust the intent of the person who made them up. Put another way, or put another way by baseball announcer Vin Scully: “People use statistics the way a drunk uses a lamp post — for support, not illumination.”

    Look at these absolute whoppers

    http://world.time.com/2011/05/24/damn-statistics-top-five-false-figures-that-mean-we-get-the-world-wrong/

    So… if anyone really wants to know how theywork I’d suggest you do the following

    Read (as a minimum):

    The Drunkard’s Walk – by Leonard Mlodinow
    The Tiger That Isn’t: Seeing Through a World of Numbers – by Andrew Dilnot
    Bad Science – Ben Goldacre

    I love debate over data… 64% of the time it provides real benefit

    * please note I haven’t published the sample size or defined real or benefit

  4. patrickhadfield · · Reply

    What Mark said – if the headline had read “a quarter of those sampled…”, it’d be fine. The article doesn’t describe the survey (and I couldn’t find it on the Orion website – but I didn’t look hard…), but I would question their statistical method.

  5. Mark Eltringham · · Reply

    I think they got the results they wanted. But most of these surveys do. What would be refreshing would be for surveys to be published that don’t entirely go with the commercial interests of firms but do allow them to talk to their customers. For example, we published a report last week (and again today) which shows that wellness programmes don’t achieve anything in the US. When we publish a report that say they do, wellness firms are on it like a tramp on chips, sharing and commenting. But this is a proper survey with statistically significant results and they don’t want to know. You’d think it must give them some opportunity to discuss what is going wrong and what they can do to improve it but I think it’s a stretch too far.

  6. […] Simon Heath: A Certain Point Of View  Channeling Obi Wan Kenobi, Simon takes issue with the reporting of a study from Orion Partners in a Personnel Today article. “Data and metrics are incredibly important and powerful. They also provoke a great degree of scepticism, much of it justified. Scientific data and research is subjected to the most rigourous (and often rancourous) of peer review processes. That used for marketing or business propaganda rarely is (unless there is sufficient public interest). Articles such as the one in Personnel Today, do nothing to convert the sceptics and unhelpfully paint entire professions in a very poor light.” What do you make of Simon’s take on the way #hrdata issues are reported? I’d love to hear your views on this one! Follow Simon on Twitter. […]

  7. […] Simon Heath: A Certain Point Of View Channeling Obi Wan Kenobi, Simon takes issue with the reporting of a study from Orion Partners in a Personnel Today article. “Data and metrics are incredibly important and powerful. They also provoke a great degree of scepticism, much of it justified. Scientific data and research is subjected to the most rigourous (and often rancourous) of peer review processes. That used for marketing or business propaganda rarely is (unless there is sufficient public interest). Articles such as the one in Personnel Today, do nothing to convert the sceptics and unhelpfully paint entire professions in a very poor light.” What do you make of Simon’s take on the way #hrdata issues are reported? I’d love to hear your views on this one! Follow Simon on Twitter. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: