Automated diagnostic machines consisting of symptom checklists have been evaluated in medicine before. The results were bleak: symptom-checkers put the correct diagnosis first only 34% of the time, and had the correct diagnosis in the top three only 51% of the time.
However, when these authors published their prior study, they presented these findings in a vaccuum – despite their poor performance, how did this compare against human operators? In this short research letter, then, these authors, compare the symptom-checker performance against clinicians contributing to a sort of crowdsourced medical diagnosis system.
And, at least for awhile longer, the human-machine is superior than the machine-machine. Humans reading the same vignettes placed the correct diagnosis first 72.1% of the time, and in the top three 84.3% of the time.
With time and further natural language processing and machine learning methods, I expect automated diagnosis engines to catch up with humans – but we’re not there yet!
“Comparison of Physician and Computer Diagnostic Accuracy.”
I’ve talked about the inflammatory vs. infectious theory for diverticulitis in a couple of my national and international presentations, and I’ve talked about the evidence supporting an observation-only strategy. Until now, that evidence was mostly a single-center randomized trial from Finland, followed by an observational report from the same.
Now, this non-treatment paradigm has migrated across the little river between Scandanavia and Europe, and has been taken up by the Dutch in this latest randomized trial. In this trial, 22 clinical sites randomized 570 patients with uncomplicated diverticulitis to either admission for antibiotic treatment, or observation and disposition per clinical stability. The antibiotic chosen was amoxicillin-clavulanic acid, with ciprofloxacin/metronidazole in the case of allergy. The primary outcome was time to recovery, with secondary outcomes relating to complications and recurrence.
Of the 570 randomized, ultimately, only 528 were analyzed. A handful were lost to follow-up, and then 35 patients withdrew from the study after allocation – 22 in the observation cohort and 13 in the antibiotics cohort. There were minor, potentially confounding differences between the two cohorts analyzed – about 8% more of the antibiotics cohort suffered from mild or severe comorbid disease.
Most patients selected for observation were admitted – with only 13% managed as outpatients. With respect to the primary outcome, there was no difference between cohorts in time to recovery. There were also few statistically significant differences in secondary outcomes, although the numbers suggest a small magnitude of harm relating to observation. Complications – ongoing diverticulitis within 6 months, need for sigmoid resection, and hospital readmission – all favored the antibiotic cohort by small, non-significant amounts. Conversely, morbidity numbers generally favored the observation cohort – relating mostly to antibiotic-related adverse effects.
Regardless, the sum of benefits and harms – in the context of the other evidence – supports antibiotic-free strategies for uncomplicated diverticulitis.
“Randomized clinical trial of observational versus antibiotic treatment for a first episode of CT-proven uncomplicated acute diverticulitis”
Our digital overlords are increasingly pervasive in medicine. In many respects, the advances of computerized provider order-entry are profoundly useful: some otherwise complex orders are facilitated, serious drug-interactions can be checked, along with a small cadre of other benefits. But, we’ve all encountered its limitations, as well.
This is a qualitative descriptive study of medication errors occurring despite the presence of CPOE. This prospective FDA-sponsored project identified 2,522 medication errors across six hospitals, 1,308 of which were related to CPOE. These errors fell into two main categories: CPOE failed to prevent the error (86.9%) and CPOE facilitated the error (13.1%).
CPOE-facilitated errors are most obvious. For example, these include instances in which an order set was out-of-date, and a non-formulary medication order resulted in delayed care for a patient; interface issues resulting in mis-clicks or misreads; or instances in which CPOE content was simply erroneous.
More interesting, however, are the “failed to prevent the error” issues – which are things like dose-checking and interaction-checking failures. The issue here is not specifically the CPOE, but that providers have become so dependent upon the CPOE to be a reliable safety mechanism that we’ve given up agency to the machine. We are bombarded by so many nonsensical alerts, we’ve begun to operate under an assumption that any order failing to anger our digital nannies must be accurate. These will undoubtedly prove to be the most challenging errors to stamp out, particularly as further cognitive processes are offloaded to automated systems.
“Computerized prescriber order entry– related patient safety reports: analysis of 2522 medication errors”
The world of anticoagulation turned upside-down with dabigatran, and continued with the Factor Xa inhibitors: rivaroxaban, apixaban, and edoxaban. While RE-LY and its ilk showed, in the settings of controlled clinical trials, that these new agents were potentially superior, or at least non-inferior, to warfarin – which is best? Do we have any idea?
Unfortunately, such comparative effectiveness work is sadly lacking, and we are forced to try and glean safety data indirectly following approval. This study pools Medicare beneficiaries using the new agents for stroke prevention in the setting of nonvalvular atrial fibrillation, and attempts to observe “real world” outcomes.
The winner on stroke prevention: rivaroxaban, by a hair. The winner on bleeding: dabigatran, by a long shot, both intra-cranial and extra-cranial. Overall mortality, then, slightly favored dabigatran.
These data are retrospective and tortured by statistical matching methods, so their reliability is hardly bulletproof. What this does raise are more questions about the appropriate usage of these new agents – and further emphasizes the importance of prospectively performed patient-centered effectiveness research.
“Stroke, Bleeding, and Mortality Risks in Elderly Medicare Beneficiaries Treated With Dabigatran or Rivaroxaban for Nonvalvular Atrial Fibrillation”
Many institutions are starting to see roll-out of some sort of clinical decision-support for imaging utilization. Whether it be NEXUS, Canadian Head CT, or Wells for PE, there is plenty of literature documenting improved yield following implementation.
This retrospective evaluation looks at what happens when you don’t obey your new robot overlords – and perform CTPA for pulmonary embolism outside the guideline-recommended pathway. These authors looked specifically at non-compliance at the low end – patients with a Wells score ≤4 and performed with either no D-dimer ordered or a normal D-dimer.
During their 1.5 year review period, there were 2,993 examinations and 589 fell out as non-compliant. Most – 563 – of these were low-risk by Wells and omitted the D-dimer. Yield for these was 4.4% positivity, compared with 11.2% for exams ordered following the guidelines. This is probably even a high-end estimate for yield, because this includes 8 (1.4%) patients who had subsegmental or indeterminate PEs but were ultimately anticoagulated, some of whom were undoubtedly false positives. Additionally, none of the 26 patients that were low-risk with a normal D-dimer were diagnosed with PE.
Now, the Wells criteria are just one tool to help reinforce gestalt for PE, and it is a simple rule that does not incorporate all the various factors with positive and negative likelihood ratios for PE. That said, this study should reinforce that low-risk patients should mostly be given the chance to avoid imaging, and a D-dimer can be used appropriately to rule-out PE in those where PE is a real, but unlikely, consideration.
“Yield of CT Pulmonary angiography in the emergency Department When Providers Override evidence-based clinical Decision support”