To the Ministry's main lobby The Ministry Blog
concert setlists

13 September, 2008

Just having a laugh

A couple of weeks ago, the BBC reported the distribution of 'Britain's happiest places'; rural Wales is merriest, apparently, and Edinburgh's to be avoided.  Except it's utter rubbish.

Of course. These pseudo-scientific non-stories are nothing new, and generally best ignored, but this one actually debunks itself:

the team from the Universities of Sheffield and Manchester stress that happiness is more a product of personal circumstances than physical location. The variations they uncovered between different places in Britain were not statistically significant.
Not statistically significant. Therefore, the distribution reported was a result of pure chance: random variation.

Yet the BBC still ran the story, on the website and TV. Partly in response to Bad Science's coverage, a number of people wrote to challenge the editorial decision, and were told that it doesn't matter; it was only a light-hearted piece.

That's fine: news reporting certainly doesn't always need to be deadly serious, but it does need to be true. There may well be a place for a little light-relief, but a world-renowned news broadcaster really shouldn't be blurring the boundary between fact and downright fiction.

[Update 24/10/08: A complaint to the BBC’s Head of Editorial Complaints has been upheld. The web-published article is to be amended, so if you read it now, it may seem rather innocuous; if you're interested, the original version is archived
here.]

Posted by Ministry at 11:55 | 228 words
Comments

>Not statistically significant. Therefore, the distribution reported was a result of pure chance: random variation.

i hate to sound negative, but this is utter codswallop, although i understand that this is what most people are taught is true. i speak here as someone with post-graduate quals in statistics/econometrics and who made a good living for many years from quant assessment of reality.

if something is not black, that does not require that it is white. all you can say is that something is extremely unlikely to be black. so it's more likely to be non-black. not white. merely not black.

that's it.

aspirin, at statistically near-guaranteed levels of probability, does not prevent pain. therefore it has no effect on pain. right?

wrong.
the absence of near-guaranteed True, does not imply anti-True.

karl popper has a lot to answer for. all-too-many people have been seriously misled about hypothesis-testing.

Posted by Saltation at September 18, 2008 01:24 AM

Uh. It does rather invalidate my criticism if I'm going to say stupid things like that; I shouldn't hold a potentially-unqualified journalist to a higher standard of proof that someone who does have postgrad-level knowledge of stats (even if a decade old) and who should know better!
Yes, insignificance doesn't mean 'false', just 'not-proven'.

Still, I'd say it also means 'not enough justification to run the story'. I don't know whether you followed the links, but they mention that the BBC covered the non-issue at least three times, twice without the disclaimer, treating it as 'proven' and, in my opinion, undermining the public's confidence in genuine research outcomes.

Posted by NRT at September 18, 2008 08:38 AM

i find Bad Science to be a fount of bad science. yes, he plays the self-righteous outrage card very well, but he's nearly always wrong. and in so doing, and presenting himself as a valid criticiser of other's research, i believe he does more damage to the public's confidence in research than the occasional shaky journalism.

but i must apologise for the tone of my comments, here and elsewhere in that late-night burst. i had come back from the pub and foolishly decided to prowl around on t'internet for a bit. you know how it is. :)
and as a result, what one types may be couched in apparently-less friendly tones than one intended. affable "oi!"s do not translate well in the absence of non-verbal signalling. one is a drongo.

Posted by Saltation at September 22, 2008 05:42 PM
.
Site Home Tull Tour History Annotated Passion Play
.
Day in the life... Page design and original graphics © NRT, 2003