SCHROEDINGER'S CAT SCAN:

Medical breakthroughs are not always what they seem (WILLIAM REVILLE , Irish Times, 3/17/11)

In 2005, Ioannidis published two papers that shook the medical research community – one in the online journal PLoS Medicine, and the other in the Journal of the American Medical Association.

In the PLoS article Ioannidis proved mathematically that, when it is assumed that there is a modest researcher bias, commonly imperfect research techniques and a common predisposition to concentrate on exciting rather than plausible hypotheses, research findings will point in the wrong direction most of the time.

The PLoS paper predicted that 80 per cent of non-randomised studies (the most common type), 25 per cent of “gold standard” randomised trials, and 10 per cent of “platinum-standard” large randomised trials must give wrong results. This mathematical prediction corresponds to the rates at which new findings are later refuted. In his JAMA paper, Ioannidis focused on 49 of the most significant medical research findings over the previous 13 years. Forty five of these 49 claimed to have discovered effective new medical treatments. Subsequently, 41 per cent of these claims were shown to be wrong or greatly exaggerated.

Ioannidis is particularly dismissive of the value of nutritional studies, declaring that the methods used in most of these studies are naïve and not nearly discriminating enough to yield meaningful results.

The situation gets even more complicated with the discovery/development of new drugs. Now, not only is individual scientific career progression at stake but big money is involved.

The multinational pharmaceutical companies are a vital part of modern medicine and make great contributions, eg the new drugs that have successfully contained AIDS were largely due to investment in R&D of multinational “big pharma”. These companies must make money.The temptation is obvious if you have spent years of work and hundreds of millions of dollars developing a new drug, and then new research shows that it is not effective. Usually the company will bite the bullet and take the hit, but occasionally . . .

Medical journals and popular accounts of medical research are full of predictions that massive breakthroughs are imminent.

A good current example is the optimistic predictions about the many cures that will spring from embryonic stem cell research.

Follow @brothersjudd