December 15, 2004

IF IT QUACKS LIKE A DUCK...:

THE BELL CURVE: What happens when patients find out how good their doctors really are? (ATUL GAWANDE, 2004-11-29, The New Yorker)

It used to be assumed that differences among hospitals or doctors in a particular specialty were generally insignificant. If you plotted a graph showing the results of all the centers treating cystic fibrosis—or any other disease, for that matter—people expected that the curve would look something like a shark fin, with most places clustered around the very best outcomes. But the evidence has begun to indicate otherwise. What you tend to find is a bell curve: a handful of teams with disturbingly poor outcomes for their patients, a handful with remarkably good results, and a great undistinguished middle.

In ordinary hernia operations, the chances of recurrence are one in ten for surgeons at the unhappy end of the spectrum, one in twenty for those in the middle majority, and under one in five hundred for a handful. A Scottish study of patients with treatable colon cancer found that the ten-year survival rate ranged from a high of sixty-three per cent to a low of twenty per cent, depending on the surgeon. For heartbypass patients, even at hospitals with a good volume of experience, risk-adjusted death rates in New York vary from five per cent to under one per cent—and only a very few hospitals are down near the one-per-cent mortality rate.

It is distressing for doctors to have to acknowledge the bell curve. It belies the promise that we make to patients who become seriously ill: that they can count on the medical system to give them their very best chance at life. It also contradicts the belief nearly all of us have that we are doing our job as well as it can be done. But evidence of the bell curve is starting to trickle out, to doctors and patients alike, and we are only beginning to find out what happens when it does.

In medicine, we are used to confronting failure; all doctors have unforeseen deaths and complications. What we’re not used to is comparing our records of success and failure with those of our peers. I am a surgeon in a department that is, our members like to believe, one of the best in the country. But the truth is that we have had no reliable evidence about whether we’re as good as we think we are. Baseball teams have win-loss records. Businesses have quarterly earnings reports. What about doctors?

There is a company on the Web called HealthGrades, which for $7.95 will give you a report card on any physician you choose. Recently, I requested the company’s report cards on me and several of my colleagues. They don’t tell you that much. You will learn, for instance, that I am in fact certified in my specialty, have no criminal convictions, have not been fired from any hospital, have not had my license suspended or revoked, and have not been disciplined. This is no doubt useful to know. But it sets the bar a tad low, doesn’t it?


Heck, if you saw just the 'scrips your doctor was on you'd run screaming into the night.

MORE:
-Left Behind: The stories are scary: A patient finds that his surgeon left a sponge or maybe a clamp in his body. But Atul Gawande is trying to write happier endings. (Douglas Starr, December 5, 2004, Boston Globe Magazine)

Posted by Orrin Judd at December 15, 2004 10:09 AM
Comments

An old joke:

Q: What do you call the student who graduates at the bottom of his medical school class?

A: Doctor.

Posted by: Mike Earl at December 15, 2004 10:40 AM

But see this, and this and this.

Posted by: David Cohen at December 15, 2004 10:49 AM

David:

Even selected cohorts fall into Bell curve patterns, no? Doctors aren't all equally qualified are they, any more than the self-selected population of lawyers?

Posted by: oj at December 15, 2004 10:57 AM

Further to David's links:

Are these results normed for patient backgrounds?

I'm willing to bet there are significant differences between Cincinatti's population, and that of whichever Minnesota hospital was also cited.

Posted by: Jeff Guinn at December 15, 2004 11:43 AM

Jeff:

Yes, but wouldn't the fact that your doctors are likely to see less of the disease you have be important to you?

Posted by: oj at December 15, 2004 12:03 PM

In any population, for any given metric, some individuals will have less, some will have more and most will be in the middle -- unless it is the metric selected for.

Medical schools don't select for skill as a doctor, but rather for proxies, so some Doctors will be more effective and some will be less, and the great majority will fall somewhere around average. But it is a huge and unjustified leap to say that large differences in outcome can only be explained by relatively small differences in effectiveness.

Posted by: David Cohen at December 15, 2004 12:13 PM

Yes, the differences are likely large too.

Posted by: oj at December 15, 2004 12:20 PM

OJ:

It isn't whether the doctors see more or less of it, but rather that certain populations, for all kinds of reasons, will have different outcomes even given identical care.

Unless the study corrected for that--and given the few minutes I had to doe some investigation, nothing suggested it did--then the findings are, at the very least, suspect.

Posted by: Jeff Guinn at December 16, 2004 8:07 AM

Jeff:

So care is identical everywhere and outcomes are entirely patient dependent? Know any doctors?

Posted by: oj at December 16, 2004 8:15 AM

Actually the Boston Globe story is more interesting. It underlines the notion that malpractice is not an effective paradigm for reducing errors and ommissions and that we should ditch it in favor of more cost effective approaches.

Posted by: Robert Schwartz at December 16, 2004 10:56 AM

OJ:

So care is identical everywhere and outcomes are entirely patient dependent? Know any doctors?

Did I say anything even remotely like that?

Posted by: Jeff Guinn at December 16, 2004 12:38 PM

"It isn't whether the doctors see more or less of it, but rather that certain populations, for all kinds of reasons, will have different outcomes even given identical care."

Posted by: oj at December 16, 2004 12:44 PM

OJ:

"... will have different outcomes even given identical care."

Words are a valuable addition to sentence meaning. All of them. "... even given ..." clearly invokes a hypothetical, which is in stark contrast to your So care is identical everywhere ...

Posted by: Jeff Guinn at December 17, 2004 12:13 PM

So you just have no point?

Posted by: oj at December 17, 2004 12:57 PM

OJ:

No, I have a point--the study was failed to take into account population distinctions. (As it turns out, the author of one of the links David cited above emailed me, and said the study was even worse in other ways than the one I identified).

There is a report in todays Detroit paper noting how the infant mortality rate among African Americans in Pontiac, MI is more than four times the rate of the rest of Oakland County.

Now you could, as the study did, attribute that entirely to the competence of doctors.

The alternative would be to suggest that it isn't at all beyond the realm of possibility that some, perhaps substantial, part of that difference is due to population distinctions beyond the reach of medical science.

In other words, the unquestioning acceptance of the former conclusion would be to fall prey range restriction error.

And, interestingly enough for a conservative, conclude that outcomes in life have nothing to do with personal choices.

Posted by: Jeff Guinn at December 19, 2004 7:35 AM

Jeff:

It's all choice--choice of doctors, which is Gawande's point. If you were having a kid would you have it in the inner city or the upscale neighborhood?

Posted by: oj at December 19, 2004 8:10 AM

OJ:

You could swap the two sets of doctors, the outcomes would remain largely unchanged, and I would still choose the upscale neighborhood.

Posted by: Jeff Guinn at December 19, 2004 3:20 PM

Jeff:

Your beluef that all doctors are equally competent is touching, if naive.

Posted by: oj at December 19, 2004 4:33 PM
« ARBITRAGE NATION: | Main | HABIT FORMING: »