Have you ever felt like you're
missing the point when reading a paper in a medical
journal? If it's the statistics that are bogging you
down, you're not alone.
A September 5 JAMA study
found that medical residents didn't do too well on the
multiple-choice biostatistics test. In fact, they did
downright poorly: the average score was a dismal 41.4%.
See below to find out how you'd do on the test.
THIS
IS A TEST
The study, led by Dr Donna Windish of the Yale School
of Medicine, tested internal medicine residents. Ten
faculty and researchers trained in clinical investigation
also took the test.
"We did a literature review in
six journals and summarized the commonly used types
of stats in the test we gave residents," says Dr Windish.
"We kept the questions clinically oriented."
The results were somewhat shocking.
Residents' average — or mean, statistically speaking
— score was 41.4% compared to 71.5% for faculty.
"We didn't expect as low scores as we found," says Dr
Windish.
"Understanding the literature is
a cornerstone of what we do as physicians," says Dr
Windish. "We all need to decide whether the latest study
is so compelling that we should alter our practice."
Evidence-based research summaries
and clinical guidelines aren't available for every situation
or patient you'll come across, and sometimes you have
to hit the books and take a look at the original literature.
And residents surveyed agreed stats were important for
this — 95% of them thought you need some stats
training to interpret the literature.
SCHOOL'S
OUT
Discouraging findings like those in the JAMA
study point to the need for a new approach to medical
education, say experts. "We need to address it at all
levels of teaching, reinforcing this knowledge across
the board," Dr Windish says.
Dr Windish suggests that doctors
at teaching hospitals are the obvious place to start.
If attending physicians put evidence-based medicine
into practice, so too will residents.
But, strangely, the study found
residents near the ends of their training did worse
than ones just out of med school.
BACK
TO BASICS
Luckily for practising doctors, knowing the definition
of a c2 test isn't necessarily the be-all and end-all
of good medicine.
Dr Gordon Guyatt, a member of the
department of Clinical Epidemiology & Biostatistics
at McMaster and the man credited with coining the term
"evidence-based medicine" (EBM), explains, "In terms
of what most people think of as biostatistics, very
little is required to practice EBM."
But certain concepts are crucial
in everyday practice and in parsing marketing material.
"Clinicians need to understand the statistical reasoning
and interpret effect magnitudes, allowing intelligent
trade-offs between one intervention and another," says
Dr Guyatt. "We express these sorts of things in terms
of relative and absolute risks. It's also important
for physicians to understand variability or uncertainty."
And perhaps, in fact, they do.
In the JAMA study, residents scored better on
some aspects of the test than others. Common stats,
such as relative risk, came more easily than more arcane
ones like, say, Kaplan-Meier analysis.
|
NRM Quiz
Do you know your stats
A new JAMA study found many
medical residents don't understand statistics.
Do you? Tally up your score to find out where
you stand.
1. Systematic error of
a study's design, conduct or analysis that causes
you to mistakenly estimate an exposure's effect
on the risk of a disease is called:
(a) Confound
(b) Bias
(c) Interaction
(d) Stratification
2. The age of a study's
participants was 26 years +/- 5 years (mean +
standard deviation). Which statement's most correct?
a) It's 95% certain the population
mean falls between 16 and 36 years
b) Roughly 95% of patients are aged between 16
and 36 years
c) No patients are younger than 16 or older than
36
3. In a diabetes detection
program, the blood sugar cut-off level for test
A is set at 130mg/100mL, while for test B it's
160mg/mL. This means:
a) Test B has greater sensitivity
than test A
b) Test B has greater specificity than test A
c) Both tests have the same sensitivity and specificity
d) Test B will give more false positives than
test A
4. A study looks at the
link between major depression and BMI. The odds
ratio for depression in subjects classified as
overweight is 0.86. Which answer's true?
a) Overweight individuals' odds
of having major depression are 14% lower than
normal-weight individuals' odds of having major
depression
b) Overweight individuals' odds of major depression
are 14% higher than normal-weight individuals'
c) An overweight individual's probability of major
depression is 0.86
d) An overweight individual's odds of major depression
are 0.86
5. A study's designed
to compare a new drug with placebo by measuring
cardiovascular deaths. Researchers figure out
they'd need 200 patients in each group in order
to detect a 15% difference in CV end points, given
90% power and a significance level of 0.1. Which
tweak would require them to boost the sample size?
a) if they want to detect a
difference of 20%
b) If they specify a power of 80%
c) If they set significance to 0.5
d) If they hope to detect a difference of 10%
6. Investigators looked
at whether systemic inflammation predicts cardiovascular
disease in women. Blood CRP was measured annually.
The table below shows the relative risk estimates
of a cardiovascular event within 5 years, according
to CRP quintile. The first quintile is used as
the reference category.
Based on the relative risk data
table, we can rest assured that:
a) For women in the first quintile,
there's no cause for concern about CV event
b) Decreasing CRP levels seems to boost risk of
CV event
c) Increasing CRP levels seems to boost absolute
risk of CV event
|
CRP Quintile
|
CRP range
|
Relative risk
|
# of subjects
|
|
1
|
0.49 mg/dl
|
1.0
|
6,000
|
|
2
|
>0.49-1.08 mg/dl
|
1.8
|
6,000
|
|
3
|
>1.08-2.09 mg/dl
|
2.3
|
6,000
|
|
4
|
>2.09-4.19 mg/dl
|
3.2
|
6,000
|
|
5
|
>4.19mg/dl
|
4.5
|
6,000
|
d) There is no association
between CRP and CV event
Answer: 1 B, 2 B, 3 B, 4 A,
5 D, 6 C
How did
you do?
0-2
correct You're worse than your dimmest
resident! Grab a copy of Statistics for Dummies—stat.
3-4
correct Regression analysis indicates
you fall within one standard deviation of the
mean. That is to say, you may not have understood
the last sentence but you manage to get along
just fine in your practice.
5-6
correct Congratulations — you're
a veritable Blaise Pascal, a modern-day Sir Isaac
Newton. Please, we implore you: put your knowledge
to good use and help out some of your colleagues.
Questions adapted from JAMA,
Sept 5, 2007, Vol 298, No 9
|
|
