Better in every group
A hospital compares two treatments for an illness. Every patient’s case is recorded as mild or severe, because severe cases are much harder to cure whichever treatment is used.
Treatment A cured 90% of its mild cases and treatment B cured 80% of its mild cases, so A did better on mild cases. Treatment A cured 40% of its severe cases and treatment B cured 30%, so A did better on severe cases too.
Cure rates, in percent. Within the mild cases A is ahead, 90% to 80%, and within the severe cases A is ahead, 40% to 30%.
The counts behind the rates
A percentage hides how many patients it is out of. Treatment A was given to 10 mild cases and cured 9 of them, which is %. It was given to 100 severe cases and cured 40, which is %.
Treatment B was used the other way around. It was given to 100 mild cases and cured 80, which is %, and to 10 severe cases and cured 3, which is %.
Each entry is patients cured out of patients treated. A treated 10 mild cases and 100 severe ones; B treated 100 mild cases and 10 severe ones.
Added up, the order reverses
Now combine the two groups for each treatment. Treatment A cured 9 + 40 = 49 of its 10 + 100 = 110 patients, and %. Treatment B cured 80 + 3 = 83 of its 100 + 10 = 110 patients, and %.
So A has the higher cure rate for mild cases and for severe cases, and the lower cure rate overall, 45% against 75%. Nothing in the arithmetic is wrong. A trend that holds in every group and reverses when the groups are combined is called Simpson’s paradox.
Adding across each row gives A 49 cures out of 110, about 45%, and B 83 out of 110, about 75%.
Overall, B’s cure rate is higher: 75% against A’s 45%.
Why: each total is weighted by its mix of cases
An overall rate is not the average of the two group rates. It is a weighted average: each group counts in proportion to how many patients are in it. Treatment A’s overall rate is (10 × 90% + 100 × 40%) ÷ 110. The 100 severe cases outweigh the 10 mild ones, so A’s total, 45%, sits close to its severe rate, 40%.
Treatment B’s mix is the reverse. Its overall rate is (100 × 80% + 10 × 30%) ÷ 110, and its 100 mild cases pull the total, 75%, close to its mild rate, 80%.
So the two totals are not comparing the treatments at all. They are comparing A’s mostly severe cases, where every rate is low, with B’s mostly mild cases, where every rate is high.
Treatment A: the overall rate, 45%, lies between the severe rate, 40%, and the mild rate, 90%, and much nearer 40%, because 100 of A’s 110 cases were severe.
Treatment B: the overall rate, 75%, lies between 30% and 80%, and much nearer 80%, because 100 of B’s 110 cases were mild.
A wins both rows and A% > B% overall, because the two mixes are close
Make A better in both rows and worse overall.
Each row is one treatment’s 110 cases, mild on the left and severe on the right, and the shaded height of each block is its cure rate. The rates never change: 90% and 40% for A, 80% and 30% for B. The handle sets how many of A’s cases are mild, and B gets the opposite mix. Drag it to the left. At 40 mild cases or fewer, A’s overall rate falls below B’s, though A still wins both groups.
What to conclude
Compare within the groups. A patient with a mild case is more likely to be cured by treatment A, and so is a patient with a severe case, so treatment A is the better treatment, whichever kind of case the patient has.
A fair overall comparison gives both treatments the same mix of cases. Give treatment B treatment A’s mix, 10 mild and 100 severe: it would cure 10 × 80% = 8 mild cases and 100 × 30% = 30 severe ones, 38 out of 110, about 35%. On the same mix, A’s 45% beats B’s 35%.
The general lesson: when two groups are made up differently, an overall total can point the wrong way. Before trusting a combined comparison, look for a factor, like the severity of a case, that affects the outcome and is spread unevenly between the groups, and compare within each level of it.
Worked example: Two Clinics Where One Recovers More of Both Kinds of Case and Fewer Cases in All
Question Two clinics treat the same illness, and every case is recorded as mild or severe. Clinic A treated 200 mild cases, of which 160 recovered, and 50 severe cases, of which 10 recovered. Clinic B treated 50 mild cases, of which 45 recovered, and 200 severe cases, of which 60 recovered. (a) Find the recovery rate of each clinic for mild cases, for severe cases and for all of its cases. (b) Which clinic would you send a patient to, and explain how one clinic can recover a greater share of both kinds of case and still have the lower overall rate.
1.Mild cases: clinic A recovered 160200 = 80% and clinic B recovered 4550 = 90%. Clinic B is ahead.
Mild cases: clinic A recovered 160200 = 80% and clinic B recovered 4550 = 90%. 2.Severe cases: clinic A recovered 1050 = 20% and clinic B recovered 60200 = 30%. Clinic B is ahead again.
Severe cases: clinic A recovered 1050 = 20% and clinic B recovered 60200 = 30%. 3.(a) All cases: clinic A recovered 160 + 10 = 170 of its 250 patients, which is 170250 = 68%, and clinic B recovered 45 + 60 = 105 of its 250, which is 105250 = 42%. So clinic B leads on mild cases and on severe cases, and trails badly overall.
(a) Over all its cases clinic A recovered 170250 = 68% and clinic B recovered 105250 = 42%, the opposite way round. 4.The reason is the mix. Severe cases recover far less often at either clinic, and clinic B treats 200 of them against clinic A's 50, while clinic A's work is 200 mild cases against clinic B's 50. Each overall rate is pulled toward the rate of the kind of case that clinic mostly sees.
The mix explains it: severe cases recover far less often, and clinic B treated 200 of them against clinic A's 50. 5.(b) Send the patient to clinic B, because whichever kind of case the patient turns out to be, clinic B recovers a greater share of them. Check by giving clinic B clinic A's mix of cases: 200 mild at 90% is 180 recoveries and 50 severe at 30% is 15, a total of 195 out of 250, which is 78%, well above clinic A's 68%.
(b) Choose clinic B. Given clinic A's mix of cases it would recover 180 + 15 = 195 of 250, which is 78%, well above clinic A's 68%.
Answer: (a) clinic A recovered 80% of its mild cases, 20% of its severe cases and 68% of all its cases; clinic B recovered 90%, 30% and 42%; (b) clinic B, because it recovers a greater share of both kinds of case, and its overall rate is lower only because 200 of its 250 cases were severe while 200 of clinic A's were mild, and on clinic A's mix of cases clinic B would recover 78%
Common mistakes
- Finding a clinic's overall rate by averaging its two group rates, as 90 + 302 = 60% for clinic B. That would only be right if the clinic treated the same number of mild and severe cases. Clinic B treated four times as many severe cases, so its overall rate must be counted from the totals, 105250 = 42%.
- Taking the overall figures as the answer because they use the most patients. The two clinics are not seeing the same kind of work, so the overall figures compare clinic B's severe cases with clinic A's mild ones. A comparison is only fair within a group, and clinic B wins both groups.