Jul 5, 2021
CanadiEM Journal Club E04 Systematic reviews and meta
analyses show notes
Welcome back to Journal Club by
CanadiEM! In this episode we go over an approach to systematic
reviews and meta analyses based on Oxford centre of EBM, and learn
about diagnosing pneumothorax with ultrasound vs X-ray
Using the Oxford centre of EBM
tool, we will ask:
- What question(s) did the
systematic review address?
- Is it likely that important,
relevant studies were missed?
- Were the criteria used to
select articles for inclusion appropriate?
- Were the included studies
sufficiently valid for the type of question asked?
- Were the results similar from
study to study?
- What were the
results?
- What is the clinical
significance of the results?
and then a clinical pearl on
pneumothorax!!
Hosts:
- Dakoda Herman
- Jayneel Limbachia
- Jake Domm
Paper: “Chest
ultrasonography versus supine chest radiography for diagnosis of
pneumothorax in trauma patients in the emergency department”
Cochrane Database of Systematic Reviews by Chan KK, Joo DA, McRae
AD, Takwoingi Y, Premji ZA, Lang E, Wakai A
What question(s) did the systematic review
address?
P: Trauma patients in the
ER
I: chest ultrasonography by non
rad physicians
C: Chest xray
O: diagnosis of pneumothorax,
improved patient safety
- Secondary: investigate potential sources of
hetero such as type of CUS operator, type of trauma, type of US
probe on test accuracy
T: inception to 10 April
2020
Is it unlikely that important, relevant studies were
missed?
- This study included prospective, paired
comparative accuracy studies in which patients were suspected
of having pneumothorax. Patients must have undergone both CUS by a
frontline non-radiologist and CXR, as well as CT of the chest or
tube thoracostomy as the reference standard.
- The authors carried out systematic searches in
the following electronic databases: Cochrane Database of Systematic
Reviews; Cochrane Central Register of Controlled Trials; MEDLINE;
Embase; Cumulative Index to Nursing and Allied Health Literature
(CINAHL) Plus; Database of Abstracts of Reviews of EIects; Web of
Science core collection (which includes: Science Citation Index
Expanded; Social Sciences Citation Index; Arts & Humanities
Citation Index; Conference Proceedings Citation Index - Science;
Conference Proceedings Citation Index - Social Sciences &
Humanities; and Emerging Sources Citation Index; and
Clinicaltrials.gov from database creation to April
2020.
- The authors also handsearched reference lists
of included articles and reviews, retrieved via electronic
searching, for potentially eligible studies. Additionally, they
carried out forward citation searching of relevant articles in
Google Scholar and looked at the “Related articles” on
PubMed.
- They did not limit the search to Englsih
language only and included articles published in all
languages.
- Their search strategy in volved the use of MeSH
terms such as Pneumothorax, Radiography, Ultrasonography, and
focused assessment with ultrasonography for trauma. They also used
many text words.
- Using this search strategy 3473 records were
identified. 1180 duplicated records were removed, leaving 2293
records to be screened. These records were screened by two of the
authors for their relevance, when there was a discrepancy a third
author decided whether to include the record or not. 2268 records
were excluded, leaving 25 full-text articles that were assessed for
eligibility. 12 studies were excluded - 5 missing CUS/CXR/CT
chest/chest tube, 4 CUS not performed by frontline non-radiologist
physicians, 2 wrong patient population, 1 wrong study design. A
total of 13 studies were included in qualitative and quantitative
analysis. 9 studies using patients as units of analysis included in
primary analysis. 4 studies using lung fields as units of analysis
included in secondary analysis.
- Authors provide a nice figure depicting
this.
- Authors did not contact experts for unpublished
data but were very thorough and transparent in their search
strategy. I think it is unlikely that important, relevant studies
were missed.
Were the criteria used to select articles for inclusion
appropriate?
- The authors of this study clearly outlined
their study inclusion and exclusion criteria.
- They included prospective, comparative accuracy
studies in which patients were suspected of having
pneumothorax.
- They included trauma patients in the emergency
department setting. Patients must have undergone both CUS by
frontline non-radiologist physicians and CXR as index tests, as
well as CT of the chest or tube thoracostomy as the reference
standard. The two main index tests were CUS completed by a
frontline nonradiologist physician and CXR, both being performed in
the supine position. If data on specific CUS findings (such as the
absence of lung sliding, absence of B-lines or comet-tail artefact,
presence of lung point, and absence of lung pulse) were available,
they planned to assess the diagnostic accuracy of these individual
CUS findings. The target condition was traumatic pneumothorax of
any severity. They defined a pneumothorax identified on CT scan of
the chest or via clinical findings of a rush of air or bubbling in
a chest drain after tube thoracostomy as the reference
standard.
- The authors excluded studies involving
participants with already diagnosed pneumothorax (i.e. case-control
studies); studies involving participants with nontraumatic
pneumothorax; studies involving participants who had already been
treated with tube thoracostomy; and studies in which a frontline
non-radiologist physician did not perform CUS.
- These criteria were appropriate for inclusion.
However, many studies did differ in their units of measurement -
lung fields vs. patients, which could introduce significant bias
into the results of individual studies.
Were the included studies sufficiently valid for the
type of question asked?
- The authors used the QUADAS-2 tool to assess
risk of bias and the applicability of each included
study.
- This tool assesses risk of bias in four
domains: patient selection; index tests; reference standard; and
flow and timing. In addition, they examined concerns about
applicability in the first three domains. They tailored the tool to
their review question - one of the signalling questions in the
patient selection domain was not applicable because they excluded
case-control studies; therefore they deleted this question from the
tool.
- Two review authors performed the assessments
independently. They discussed and resolved any disagreements that
arose through consultation with a third review author.
- They included a figure describing their
assessments of study quality.
- Of the nine studies that we included in the
primary analysis:
-
- One had a low risk of bias, two had an unclear
risk, and six had high risk of bias in the patient selection
domain, mostly due to convenience sampling or inappropriate
exclusion criteria, such as excluding haemodynamically unstable
patients, lack of access to CUS, chest wall injuries precluding
CUS, or if CT was not indicated.
- The risk of bias in the interpretation of CUS
results was low in five studies, unclear in two studies, and high
in two studies; this was related to unclear blinding methodology of
outcome assessors interpreting CUS and CXR
results.
- The risk of bias in the interpretation of CXR
results was low in two studies, unclear in six studies, and high in
one study; this was largely due to unclear blinding methodology of
outcome assessors interpreting CXR and CT results, as in some
studies it was not clear whether radiologists had access to both
imaging results or not.
- The risk of bias introduced in interpretation
of the reference standard results was low in three studies but
unclear in the remainder for similar concerns regarding blinding
methodology.
- The risk of bias in the flow and timing domain
was low in two studies, unclear in four studies, and high in three
studies; this was due to the exclusion of patients based on missing
CT data or unclear/inappropriate time intervals between CUS, CXR,
and CT.
- They judged applicability concerns regarding
patient selection as low for six studies but high for three
studies; this was due to the exclusion of haemodynamically unstable
patients or lack of access to CUS despite the study focusing on
comparing CUS.
- They judged one included study to have unclear
concern regarding applicability of the reference standard used as
there was insuIicient reporting of the method of
assessment.
- They deemed all other domains for applicability
concerns as low risk for all studies.
- Studies that used lung fields as their unit of
analysis had several limitations including missing CUS data for
some lung fields and using two CUS tests (one for each lung field)
compared to one CXR (for both lungs) on the same patient.
Inherently, there would be an inability to blind the CUS operator
during collection of CUS data while performing the two CUS tests
(one on each side of the patient), as well as during the
interpretation of the CXR and CT results between the two lung
fields. By analysing lung fields separately, it is diIicult to
ascertain whether patient characteristics, past medical history, or
traumatic injury pattern could have affected one or both lungs and
may have confounded the diagnostic accuracy. Out of the four
studies included in the secondary analysis:
-
- The risk of bias in the patient selection
domain was low in one study, unclear in two studies, and high in
one study; this was due to inappropriate exclusion criteria, such
as excluding haemodynamically unstable patients or chest wall
injuries precluding CUS, or due to unclear sampling
technique.
- The risk of bias introduced in interpreting CUS
results was low in two studies and unclear in two studies due to
lack of clarity about blinding of the outcome assessors
interpreting CUS and CXR results.
- The risk of bias introduced in interpreting CXR
and CT results to be low in one study, unclear in two studies, and
high in one study; this was again due to definite unblinded
interpretation of test results or unclear blinding methodology of
outcome assessors interpreting CXR and CT results.
- The risk of bias in the flow and timing domain
was low in two studies and high in two studies due to missing
patient data.
- They judged applicability concerns in the
patient selection domain as low for two studies, unclear for one
study, and high for one study, due to unclear patient selection
methods and exclusion of haemodynamically unstable patients or
chest wall injuries precluding CUS.
- They judged one study to have unclear concern
regarding applicability of the reference standard, as blinding of
the outcome assessor interpreting the results of CUS, CXR, and CT
was unclear. We deemed all other domains as 'low concern' for all
studies.
- The authors were thorough in their assessment
of each study's quality and discussed how this may have impacted
the results. The most common area of bias seemed to be in patient
selection where 11/13 studies were judged to be at unclear or high
risk of bias.
Were the results similar from study to
study?
- Substantial heterogeneity in sensitivity
analysis of supine CXR
-
- Based on Figure 4, Forest
plot:
-
- Sensitivity ranged from 0.09 to
0.75
- Wide and non overlapping confidence intervals
are suggestive of high variability between
studies
- Limits the evidence
- But they do not list the reasons for why such
heterogeneity exists
-
- Some possibilities for the heterogeneity
include the high or unclear risk of bias of included
studies
- There is no I squared statistic shown - a
statistical measure that confirms the presence of significant
heterogeneity in a meta-analysis
Results of the study:
- Included 13 studies, 1271 trauma patients with
410 who had a pneumothorax
-
- 9 studies used patients as unit of
analysis
- 4 studies used lung field as unit of
analysis
- Most studies were high or unclear risk of bias
- 11/13
- CUS sensitivity and specificity: 0.91 (95% CI:
0.85 - 0.94) and 0.99 (95% CI: 0.97 - 1.00)
- CXR sensitivity and specificity: 0.47 (95% CI:
0.31 - 0.63) and 1.00 (95% CI: 0.97 - 1.00)
-
- Difference in sensitivity: 0.44 (95% CI: 0.27 -
0.61)
Practise Changing?
This is an excellently executed
Systematic Review that makes the most of the limited evidence
available and presents such in a transparent way. The results of
this study suggest that CUS has better sensitivity in detecting
pneumothorax in the emergency department than CXR, and comparable
specificity. This publication adds to the body of research that is
building in support of US in the primary care setting, and promotes
a change in culture as US adoption rates grow.
Clinical pearl:
Pneumothorax, or, air in the
pleural space, is a fairly common complication of thoracic traumas,
occurring 15-50% of the time.
- History: If you’re able to get a history, and
often you won’t be able to, you’ll hear about a Sudden, severe,
chest pain, sometimes referred to shoulder and maybe associated
pleuritic pain or SOB.
- On Exam: Exam findings range from nothing to
respiratory distress and shock. May have decreased chest wall
motion, subcutaneous emphysema, or poprock skin, hyperresonance, or
decreased or absent breath sounds on the affected
side.
- Imaging: traditionally via x-ray or US
depending on physician.
-
- On xray: Absence of lung markings beyond the
edge of the lung. The edge of the scapula or upper ribs are often
mistaken for a pneumothorax.
- On US: Scan over the most upright part of the
patient's chest. if supine go right over the top of their chest
(third or fourth intercostal space in the midclavicular line and is
repeated on both sides) and if upright scan the apices. Looking for
lung sliding and the classic “seashore sign” in M-mode in normal
lungs, or absence of lung sliding and “barcode sign” seen in
pneumothorax. There are tons of good videos online to take a look
at.
- CT is gold standard, but rarely
necessary
- Rush of air on thoracostomy is
also diagnostic.
- Treatment: varies based on
severity, from no treatment if patient tolerating well, to drains
and chest tubes, to needle or finger thoracostomy for immediate
decompression if pneumothorax clinically indicated and
hypotension.