The use of torture in statistics

The recent CONCORD report on cancer survival rates has been used by some on the right to support their contention that the US health care system is superior; according to Catron, the “much-maligned U.S. health care system is the best in the world.” Catron and his fellow travelers’ use of the researchers’ findings has been, to be kind, selective at best.
The study looked at populations around the world, including 16 US states and a half-dozen metropolitan areas that represented 42% of the US population. (Not all state data was available due to collection issues and standards). Four types of cancer, including breast (women), colon, rectum and prostate, which comprise a majority of all newly diagnosed cancers in adults were examined.
From this, Catron concluded:
“Nowhere is this [the superiority of the US health care system] more obvious than in the area of cancer treatment. John Goodman reports on another study published in Lancet Oncology showing that the U.S. has the best cancer survival rates:
[this is a quote from Goodman, not from the Lancet] “A new study of cancer survival on five continents lays to rest the theory that Americans fare poorly compared to other developed countries … in almost every category Americans survive cancer at higher rates than patients in other developed countries.”
Here’s a chart showing U.S. survival rates for breast and prostate cancer compared to the survival rates of four countries whose health care systems are often touted as superior to ours”
Here are a few of the problems with Catron’s argument.

• Catron used two types of cancer, not the four that were studied. And the two he used were (surprise!) the ones that favored the US. Rectal and colon cancer survival rates were statistically similar in the U.S., Japan, Canada, France, the Netherlands, Sweden, and Australia
• There’s no doubt the US survival rate for one of his examples – prostate cancer – is very high. There’s also no doubt that this cancer is way over-hyped in the US, with much more screening and testing, lots of which identifies cases that are unlikely to be fatal. Prostate cancer is quite common among older men. It usually grows very slowly, so slowly that most of us end up dying from something else. So we’re paying way more for screening that is marginally useful from a population perspective.
• The incidence of diagnosed prostate cancer in the US is much higher than in other countries
• Catron may have meant to say that the US health care system is much better for white folks. In the US, there’s a wide disparity in survival rates between blacks and whites. For breast cancer, survival for white women in the USA (84.7%) was 14% higher than for black women (70.9%). For colorectal cancer, white patients in the US had survival rates 10% higher than for black patients (60% compared with 50%). For prostate cancer, However, there was a 7% difference in survival between black and white men (92% compared with 85.8%).
• The US stats covered less than half of the population; we don’t know if survival rates in the other 58% of the population are better/worse/the same

The leaps of logic in Catron’s argument would do credit to gymnasts with Olympian aspirations. First that the data is complete (it is not); second that survival rates for two types of cancer is a fair evaluative basis (what about other maladies?): and third that the US has a better health care system because of breast and prostate cancer survival rates.
The logical errors in Catron’s analysis, combined with his faulty arguments, would be laughable if they weren’t representative of how some actually ‘think’. Their approach to any issue is to try to scare the bejesus out of folks, ignoring data and evidence contrary to their opinions. This does them no credit, and over the long term likely undermines their attempt to influence the dialogue on health system reform.
Looking at data from the Commonwealth Fund, and employing Catron’s logic (better survival rates = better health care systems), we find that the 5 year survival rate for kidney transplant patients was 94% in Canada, 86% in the UK, and 83% in the US in 2001. Does this mean that the Canadian system is better than the US system? Of course not – and I’d bet that Catron and his ilk would be quick to point out the problems inherent in drawing such a conclusion.
What does this mean for you?
If you torture numbers long enough, they’ll say anything you want them to. Catron’s misuse of the CONCORD statistics is Exhibit One in the case against this horrifying practice.