Statistical significance tells us that an observed difference is reliable, but whether the difference is large enough to be important cannot be resolved by applying a statistical formula.
The more important recent issue seems to be a study published in The Lancet which indicates a 9% increased risk of Type 2 diabetes associated with Crestor. As usual the question is whether these risks outweigh the benefits. The Crestor trial was typical of heart disease trials and involved a large population of 18,000 subjects. As the article notes, statistical significance in the reduction of heart attacks in this population does not necessarily translate to clinical significance:
Critics said the claim of cutting heart disease risk in half — repeated in news reports nationwide — may have misled some doctors and consumers because the patients were so healthy that they had little risk to begin with.
The rate of heart attacks, for example, was 0.37 percent, or 68 patients out of 8,901 who took a sugar pill.
Among the Crestor patients it was 0.17 percent, or 31 patients.
That 55 percent relative difference between the two groups translates to only 0.2 percentage points in absolute terms — or 2 people out of 1,000.
Stated another way, 500 people would need to be treated with Crestor for a year to avoid one usually survivable heart attack. Stroke numbers were similar.
“That’s statistically significant but not clinically significant,” said Dr. Steven W. Seiden, a cardiologist in Rockville Centre, N.Y., who is one of many practicing cardiologists closely following the issue. At $3.50 a pill, the cost of prescribing Crestor to 500 people for a year would be $638,000 to prevent one heart attack.
Is it worth it? AstraZeneca has concluded it is.
“The benefit is vanishingly small,” Dr. Seiden said. “It just turns a lot of healthy people into patients and commits them to a lifetime of medication.”
Statistical Signiﬁcance Versus Clinical Importance
Trials on Exercise Therapy for Chronic Low Back Pain as Example
Maurits van Tulder, PhD,*†§ Antti Malmivaara, MD, PhD,‡ Jill Hayden, DC,§
and Bart Koes, PhD
Study Design. Critical appraisal of the literature.
Objecives. The objective of this study was to assess if
results of back pain trials are statistically signiﬁcant and
Summary of Background Data. There seems to be a
discrepancy between conclusions reported by authors
and actual results of randomized controlled trials. Little
attention has been paid to the problem of over-reporting
Methods. All 43 trials of the Cochrane review on exer-
cise therapy for low back pain were included. Descriptive
analyses were conducted.
Results. Eighteen trials reported positive conclusions
in favor of exercise. Only six of the 43 studies showed
both clinically important and statistically signiﬁcant differences in favor of the exercise groups on function, and 4 on pain.
It seems that many conclusions of stud-
ies of exercise therapy for chronic low back pain have
been based on
statistical signiﬁcance of results rather
than on clinical importance and, consequently,
may have been too positive.
Authors of trials should report not only
statistical signiﬁcance of results but also clinical importance.
Key words: low back pain, randomized controlled trials,
statistical signiﬁcance, clinical importance. Spine 2007;32: