Some (more) links

I know the previous post contained a few, but I’ve been stocking up on them for a while and too many links in one go is bad for you anyway.

1) The Rule of 72. I’ve only read a couple of post from betterexplained so far, but it looks like an interesting site.

2) How Cancer Begins. I don’t know if I’ve blogged this before, but I don’t think so. It’s a very good lecture, so even if it takes most of an hour, you should give it a chance.

3) When in doubt, shout – why shaking someone’s beliefs turns them into stronger advocates. Part of why Ed Yong’s blog is one of my favourites. But only a small part. Here’s another.

4) A time waster. If you go through the main site, they require you to sign up, and I won’t do that – and you don’t need to do it either if you come via the link provided. At the bottom of the page there are links to other ‘Popular Games’, I’ve also tried word-bubles. I probably won’t try any of the others. I guess there are worse ways to spend your time if you’re really bored, though I should probably warn those who don’t already know that these kinds of games are probably as addictive as blitz-chess. Not something you should be spending much time doing.

Stuff you should read

Dealing with the high quantity of scientific error in medicine. Many of the comments to the post are (in my opinion) uninteresting stuff about diet, but this comment is quite good, and so is Yvain’s response here. Here’s one bit from the post, Ioannidis’ corollaries:

“Corollary 1: The smaller the studies conducted in a scientific field, the less likely the research findings are to be true.

Corollary 2: The smaller the effect sizes in a scientific field, the less likely the research findings are to be true.

Corollary 3: The greater the number and the lesser the selection of tested relationships in a scientific field, the less likely the research findings are to be true.

Corollary 4: The greater the flexibility in designs, definitions, outcomes, and analytical modes in a scientific field, the less likely the research findings are to be true.

Corollary 5: The greater the financial and other interests and prejudices in a scientific field, the less likely the research findings are to be true.

Corollary 6: The hotter a scientific field (with more scientific teams involved), the less likely the research findings are to be true.”

…

It looks like this article that the first article I linked to above links to as well is well worth reading too. It takes on, among other things, the subject of meta-studies:

“Now let’s iterate this [...publication/attempted replication] process several times. Every couple of years, some enterprising young investigator will decide she’s going to try to replicate that cool effect from 2009, since no one else seems to have bothered to do it. This goes on for a while, with plenty of null results, until eventually, just by chance, someone gets lucky (if you can call a false positive lucky) and publishes a successful replication. And also, once in a blue moon, someone who gets a null result actually bothers to forces their graduate student to write it up, and successfully gets out a publication that very carefully explains that, no, Virginia, lawn gnomes don’t really make you happy. So, over time, a small literature on the hedonic effects of lawn gnomes accumulates.

Eventually, someone else comes across this small literature and notices that it contains “mixed findings”, with some studies finding an effect, and others finding no effect. So this special someone–let’s call them the Master of the Gnomes–decides to do a formal meta-analysis. (A meta-analysis is basically just a fancy way of taking a bunch of other people’s studies, throwing them in a blender, and pouring out the resulting soup into a publication of your very own.) Now you can see why the failure to publish null results is going to be problematic: What the Master of the Gnomes doesn’t know about, the Master of the Gnomes can’t publish about. So any resulting meta-analytic estimate of the association between lawn gnomes and subjective well-being is going to be biased in the positive directio. That is, there’s a good chance that the meta-analysis will end up saying lawn gnomes make people very happy,when in reality lawn gnomes only make people a little happy, or don’t make people happy at all.”

Some meta-analysts are more aware of the publication bias problem than others – I remember reading a meta-study by Martin Paldam a while ago where he emphasized this problem in the analysis, and I believe he’s actually done a meta-study on publication bias as well, though I don’t remember which subject it was about and I’m too lazy to look it up now. In some studies this issue is hardly even mentioned though.

If you read the links, you’ll become much better able to evaluate some of the stuff that’s out there. Btw. I am somewhat in agreement with Yvain when it comes to two main points: a) If it is indeed true that a lot of the stuff that gets published in medical journals later turn out to be wrong, the most likely explanation is that ‘the system’ is generally working and that we’re getting smarter over time, and b) the fact that the findings of ‘mainstream’ researchers are more prone to error that you might have thought does not make the non-mainstream people any less unlikely to be wrong.