Christmas message and update on EMJ.

Below is a short letter from Ellen sent out to the editors on the journal. It outlines where we came from, what we’ve done and where we are going. Although originally intended for the internal team I really felt this encapsulated how Ellen has led us to improve, develop and future proof the journal. With her permission I have reproduced it here.

Ellen Weber

From Ellen Weber. Editor in Chief. Emergency Medicine Journal

A holiday message from the Editor of the Emergency Medicine Journal

As 2017 closes, I wanted to take a moment to look back and reflect on what we have accomplished at EMJ and a bit about my vision and hopes for the coming years. We could not have gotten where we are without such a dedicated and talented editorial and publishing team, and the credit belongs to you.

There were two questions that I was asked when I started as editor of the EMJ. At my interview, I was asked: Who is the journal for – the author or the reader? At my first appearance at the CEM meeting, I was asked if the EMJ needed to be more selective.

These questions have helped to shape my vision of what I’ve hoped we can accomplish with the EMJ. First is to raise the quality of the articles we are publishing, and secondly to make the journal more relevant to readers. This continues to be my vision for the journal. To provide a high quality journal, that readers look forward to receiving, and which publishes research that has an impact on policy, practice and patient and physician well-being.

Here’s what we’ve accomplished so far:

Quality: We have set several editorial standards that have helped to improve the quality of the papers, while at the same time providing clearer guidance for authors so that their submissions are more likely to meet these quality standards.

All research papers must now be submitted with a research checklist. This helps to ensure that the papers contain all the necessary information on methods so we can evaluate them fully.

Statistical review: We have developed a cadre of statisticians who review almost all of the original articles before they are published. The statistical reviews have raised the quality of our papers immensely. It is extremely rare that a statistician does not find a major issue with the statistical analysis in the paper, despite prior content reviews.

Standards have been gradually implemented. We began with requiring that all clinical trials must be prospectively registered to be considered for publication, as recommended by the ICJME. We then added a requirement that uncontrolled before and after studies include an interrupted time series to account for secular trends. Authors are instructed to use confidence intervals instead of p values; sample size calculations must be included; surveys must have a clear response rate; chart reviews must report inter-rater reliability. We have essentially stopped publishing audits.

We are far more selective about what we publish; papers need to have generalizability, international relevance, clear implications for practice, an unbiased interpretation of the results, and an honest discussion of limitations. Our acceptance rate for original research articles is about 20%, which is typical of higher quality journals.

To educate the authors, we have published several editorials explaining our policies, and have included on our website a section that provides additional guidance on meeting our standards. The editor recorded 4 videos for the website to help authors understand what was needed in their articles and has spoken at a number of conferences to provide authors with guidance on how to write their manuscripts.

Our Impact Factor has improved, from 1.64 in 2013 1.861 in 2016 (2017 figures are not yet out).

Reader Interest and Relevance
While many of our readers are researchers as well, by far the majority are practicing physicians who may find it difficult to see how original research articles will affect their practice. We’ve addressed this in two ways: one, doing more to put the articles in context and explain the value to the reader and 2, providing other types of content that are educational and evidence-based.

Making research more accessible:

  • We have added a box at the beginning of each article that explains what is already known on the topic (i.e. where the paper sits in current literature) and then, how the findings of the study may affect practice or add to our knowledge.
  • Primary survey – The primary survey is not new, but is used to also provide background and context for the articles we are publishing in that issue.
  • Editor’s and Reader’s Choice – We developed these to highlight two articles per month. The Editor’s choice is free to access (as is any accompanying commentary) and the Reader’s choice is selected based on downloads from the website since it was published. Both are highlighted on the cover and their relevance explained in the Primary survey. Thank you Tomasso at BMJ for continuing to adjust our covers and TOC’s to perfection!
  • We have published articles that explain statistical concepts that appear in some of our research papers.
  • Article length: To the extent possible, we have attempted to keep all research articles to 3000 words or less through careful editorial suggestions, and have expanded the use of the “short report”. Shorter articles are preferred by busy readers.
  • Topic headings: We have recently created topic headings for groups of articles published in a single issue (e.g. Paeditrics, Geriatrics, Meeting Demand for Services) to help guide the reader to articles that may have special interest in.

Non-research articles that provide perspective, education, and insight. For example:

  • We launched the Top Ten, a brief review article that allows readers to quickly obtain new information, eg.. Top Ten Apps for the Clinician, Top Ten Ways to introduce In Situ Sim; Teaching Tips, etc.
  • The View from Here – This first person narrative allows readers to learn aboutunique experiences in practicing emergency medicine, or experiences that may change how they practice. Examples have included clinical experiences in limited resource countries, working with the elderly in the EDs, being a pioneer in the early days of emergency medicine.
  • Reviews: We continue to [publish both narrative and systematic reviews. However, we have begun a new type of review – Expert Practice Review. Experts are asked to address clinical questions that are relevant to the emergency physician or prehospital practitioner at the point of care. References are minimized to only those that have formed the experts’ opinion on those specific issues. The first of these is being published in February and a second is about to be accepted. Two others are currently in progress.
  • In Perspective – While some journals do a “journal review” where they cite a half dozen papers of interest from another journal, we feel that this is not very educational for our readers. For this reason, we’ve started the In Perspective series, where an expert in the field discusses the implications of recent research. Thus far we’ve run these on chest pain diagnosis, frequent use of emergency departments, the new ATLS guidelines, the association of costs of care and outcomes. These have gotten excellent coverage on social media.
  • Innovations in Emergency Medicine – These are short reports on new ways of delivering health care or adjunctive services. They have included a visual triage method for limited resource countries and a new legal service for victims of trauma. These innovations are often difficult to test formally, but generally have some metrics showing their success.
  • Image Challenge – Images in EM was converted to a quiz to provide interactive content and self-assessment. 

Social Media

Overall in the past year, our altmetrics scores appear to have improved from prior. This is difficult to evaluate as we don’t have a “journal” score, but on browsing our website, I see more high numbers than I have in the past.

  • “Predicting outcomes in traumatic out-of-hospital cardiac arrest: the relevance of Utstein factors” Altmetrics 125
  • Relationship between non-technical skills and technical performance during cardiopulmonary resuscitation: does stress have an influence? Altmetric 131
  • Ibuprofen versus placebo effect on acute kidney injury in ultramarathons: a randomised controlled trial 14 2 (Picked up by 10 news outlets including Outside, Newsday, Medscape, Yahoo News)
  • Increased weekend mortality is not associated with adoption of seven day standards. Altmetric 245.
  • We have a monthly podcast of our primary survey..
  • We have 30 K Twitter followers.
  • A fantastic blog
  • We have an automated feed of articles when they are first released on line. 

We have clearly done a lot in terms of our quality, relevance and recognition. But we are not resting here. We have a great product. We need to ensure its sustainability and do more to let others know about what we have to offer! In the next years we will continue to work on improving our media presence, commissioning thought- provoking commentary, truly useful expert reviews. We will continue to work on shortening n turn-around times (Shout out to Princess and the EMJ team at BMJ!) and finding ways to attract high quality research papers.

Thank you all for your hard work to this point and I look forward to taking the next steps together with you.

All the best for 2018!

Ellen Weber

How Theme Park, Space Invaders and Go have paved the way for exponential healthcare

I often imagine my retired self looking back at this point in my career, marvelling at how primitive it all was. By that stage, hospital fax machines, handwritten patient notes, stethoscopes, ‘bleeps’ and other relics of a time-gone-by will be collecting dust in the Ancient Medical History Museum. I’ll be a regular visitor at the museum, chuckling at my old life. But I’ll do so remotely from the comfort of my own home with a virtual reality headset on. And I’ll be living on a spaceship.

It’s 2017, and we are standing at the precipice of the most dynamic and transformative era in healthcare history. The digital health era. New technology is poised to rampage through the status quo, radically changing the role of the clinician and the patient. Our industry is ripe for disruption, and one technology promises to provide the most exponential change – artificial intelligence (AI).

But we’ve been hearing this for ages, right? Is anywhere in the NHS applying this technology or is it all just hot air?

First, what is AI?

It’s frustratingly hard to define. Google and YouTube produce hundreds of conflicting explanations. The reason being –  intelligence itself is a confusing concept.

In simple terms, AI ‘agents’ are machines that utilise a wide range of computer science applications to solve problems previously thought only possible using ‘natural’ human cognition. There are two distinct categories – ‘narrow’ and ‘general’ AI.

Narrow AI agents tackle specific tasks. For example, weather forecasting, playing jeopardy or operating a driverless car. It involves pre-programming the machine with all the knowledge it could possibly need for task completion. Therefore, it could be argued that the ‘intelligence’ demonstrated by narrow AI resides in the brain of the human developer.

Narrow AI is already ubiquitous in the average person’s daily life via Apple’s Siri, Facebook’s friend suggestions or Netflix’s film recommendations. It has rapidly become an indispensable part of modern digital infrastructure, but each individual application is limited to its specific function.

A much greater, more complex challenge, is creating general AI, sometimes described as ‘true’ AI. This is when the machine can attempt a broad range of tasks without any prior pre-programming. It interprets and operates in its environment in a similar way to a human being. At the core of general AI is the ability to learn from scratch – ‘machine learning’.

Who are DeepMind?

General AI has remained elusive for the scientific community until relatively recently. Enter British AI company DeepMind.

The start-up was founded in 2010 by Londoners Mustafa Suleyman, Shane Legg, and Demis Hassabis (child chess prodigy and creator of wildly successful simulation game Theme Park aged 17) with the simple but ambitious mission to “solve intelligence, and use it to make the world a better place” (1).

DeepMind first hit the headlines when they designed a single algorithm that taught itself how to play and subsequently dominate forty-nine different Atari 2600 video games (2).

On first look, AlphaGo seems a narrow application of the technology, with a strikingly similar feat famously achieved nearly two decades earlier by IBM’s Deep Blue computer when it defeated chess GrandMaster Garry Kasparov (5). On closer inspection, AlphaGo and Deep Blue are very different digital creatures.

Chess is a game with a finite number of scenarios, and is therefore logic-based. Deep Blue was pre-programmed with every single possible move and outcome, and defeated Kasparov by ‘brute force’. Definitively narrow AI.

Go is also a board game, but there are more possible moves than there are atoms in the universe, making it arguably the most complex game ever devised (6). With such an unfathomable number of different possible strategies, the game goes beyond simple logic, and boils down to a battle of gut feeling and instinct. It would be impossible to pre-programme AlphaGo with every potential scenario. Instead, the machine self-learnt the game from raw experience (hundreds of hours of ‘practice’ game time), approximated expert intuition and creativity, and defeated the world’s best.

DeepMind’s algorithms uniquely combine two neuroscience-inspired machine learning strategies. The first is called ‘deep learning’. A grossly oversimplified explanation of this process is to say that artificial neural networks ingest vast quantities of raw data input, with layers of ‘neurones’ interacting as they process the information, gradually refining observations made, and eventually producing abstract ideas (7, 8).

The second strategy is ‘re-enforcement learning’. Essentially, the machine observes and interacts with its defined environment (e.g. random firing in Space Invaders, or moves in Go), learns by trial-and-error to maximise rewards/minimise penalties, gradually deepening understanding of the environment and moving itself closer to the overall goal (e.g. a perfect game score) (7, 9).

Whilst Atari games and Go are relatively simple environments, they have provided the perfect platform for demonstration of definitively general-purpose AI that truly mimics human-style learning – observation, reflection, mental model refinement and improved action. The first ‘look’ the machine has at a game of Go is directly comparable to a baby opening its eyes for the first time.

Naturally, DeepMind’s work hasn’t gone unnoticed by the tech giants. In 2015, before the company had started generating revenue, they were acquired by Google for £400 million (10). Oh, and Elon Musk is an investor (11).

DeepMind’s top priority? Healthcare.

The Ophthalmologist with a lightbulb moment

“I was a nerd. I loved computer games” says Dr. Pearse Keane, Consultant Ophthalmologist and Clinical Scientist Award recipient. “When I was kid growing up in Dublin, I used to sneak over to my neighbour’s house to play on their Atari 2600. I wanted one, but my parents wouldn’t allow it. I saved up and bought my first computer which was a Commodore 64. I started to learn to programme a tiny bit then, but to my great regret I didn’t pursue it and just went down the video games road. I remember buying Theme Park.”

Pearse KeaneAs the son of a doctor, and a high achiever in the academic arena, Dr. Keane was always destined for medical school. He graduated in 2002, and then embarked on a 13-year journey through rigorous clinical and academic ophthalmology training. He never lost his passion for technology but struggled to fit it into his busy schedule.

“I would have loved to have had the time to do a Masters in machine learning or build some software development skills,” he says. “The long path and hierarchical nature of medicine is a barrier to that”.

Today he works at Moorfields Eye Hospital in London, and subspecialises in retinal disease, with 70% of his working week spent doing research. His academic focus is retinal imaging via optimal coherence tomography (OCT – like ultrasound, but light waves are measured as opposed to the echoes of sound waves). OCT is quick, non-invasive, and produces extremely high resolution retinal images that resemble histopathology slides.

Ophthalmology is a specialty drowning in referrals. Eye clinic appointments constitute 10% of hospital appointments across the entire NHS (roughly 10 million total), second only to orthopaedics (12). Referrals have increased by a third in the last 5 years, and it’s only going to get worse as OCT scanners have been rolled out in most high-street opticians across the UK (13). In most centres, the expertise to accurately read OCT scans is usually not available, and so even the tiniest possible deviation from normal means referral to a retinal specialist.

According to Dr. Keane “it’s as if every GP in the country was given an MRI scanner, told to scan every cough or headache that comes in, but not given the training to read MRIs”. Swathes of false positive referrals crowd clinics, and increase the risk that patients with genuine sight-threatening retinal disease, like wet age-related macular degeneration (AMD), don’t get seen in a timely fashion.

“You have patients who have lost sight in one eye completely from wet AMD, and then they develop the early signs of it in their other eye,” he explains. “They’re given an urgent appointment for 6 weeks later. You can imagine how stressful that must be.”

In July 2015, Dr. Keane read an interview in Wired magazine profiling DeepMind (14). “Mustafa Suleyman – one of DeepMind’s cofounders and its Head of Applied AI – mentioned interest using AI for healthcare,” he says. “That was when I had my lightbulb moment. I tracked him down and sent him a message on LinkedIn. To my joy, he replied in a day or so, and within a few days I was meeting him for coffee.

“My idea was simply that we should apply AI, in particular deep learning, to OCT scans, and triage the scans,” he continues. “So, if you develop a sight threatening disease you can get in front of a retinal specialist within days rather than months. Similarly, if you have nothing serious wrong with you, you don’t get falsely referred in, with all the anxiety associated with that”.

In July 2016, the collaboration between Moorfields and DeepMind was formerly announced to the media, generating the kind of fanfare one might expect from the NHS, say, partnering with a commercial powerhouse like Google (15). However, research in earnest hadn’t yet begun, and the year preceding the announcement was spent meticulously packaging approximately one million anonymised, historical OCT scans so that they could be reliably fed to DeepMind’s algorithm. The biggest obstacles were predictable NHS issues like poor labelling, awkward file formatting, and multiple different types of scanner providing the images. And, naturally, there was a lengthy ethics approval process.

“We really put a lot of effort into being as transparent as we could” says Dr. Keane. “We have a section of the Moorfields website dedicated to the collaboration, with questions and answers, interviews, and how to opt out if you are a patient (16). We’ve also published our study protocol prior to starting research, which is slightly alien as normally you wait until you have the results (17).”

Research is well under way, and Dr. Keane describes the preliminary results as “very encouraging… even exciting”. Deep learning is applied in much the same as with the Atari or Go research. With Space Invaders, the AI agent was rewarded every time it fired its laser cannon and hit an alien. With retinal disease, the AI agent labels segments of OCT scans that have been pre-labelled by ophthalmologists, and gets a game-like reward if it is accurate (18).

“What we hope to do is to have an algorithm able to achieve expert performance,” says Dr. Keane. “In other words, an algorithm that can look at an OCT and diagnose any retinal disease that a retinal specialist from Moorfields could diagnose.”

The collaboration is hoping to publish their results by the close of 2017 and establish proof of concept. The aim after that would be to then perform a full, prospective randomised controlled trial to properly validate the algorithm.

Why is this a big deal?

If the results live up to the hype and prove that a machine can be rapidly trained to demonstrate human-level expertise at interpreting highly specialised imaging, it will be a monumental moment In NHS history.

But it’s not just happening at Moorfields. Similar AI-based projects are underway elsewhere in the NHS and around the world. DeepMind are also collaborating with University College London Hospital in research that explores radiotherapy planning for head and neck cancers (19), as well as developing an app with the Royal Free Hospital that identifies patients at risk of acute kidney injury (20)IBM’s Watson is being piloted as an interactive tool for improving patient engagement and understanding at Alder Hey Children’s Hospital in Liverpool (21), whilst researchers at Stanford University have trained a deep learning algorithm to diagnose skin cancers as accurately as dermatologists (22).

One can imagine as the evidence-base snowballs and clinicians increasingly ‘trust’ AI, researchers will loosen their grip on what the machines are ‘allowed’ to learn. They will excel beyond mere expert-level diagnostics, and start shedding light on new patterns and pathophysiological associations in much the same way that AlphaGo introduced new strategies to a game thousands of years old. This technology has the potential to de-shroud the countless mysteries of modern medicine, and transform what we can achieve as treating physicians, as well as what we can survive from as ailing patients.

Where I work in the Emergency Department, I see multiple opportunities where AI could pounce and turbo charge our clinical performance. More accurate interpretation of emergency radiology and electrocardiography seem like the lowest hanging fruit. We could use it to identify deteriorating patients hyper-early by continuous monitoring of vital signs in combination with other data sets such as blood results and past medical history. It could augment effective pharmacological decision-making, including antibiotic stewardship. Perhaps there is scope for AI-assisted front-of-house triage, along with a reliable ‘Dr. Google’ algorithm that will reassure well patients at home and ease pressure on primary and secondary care (23).

The human brain requires oxygen. AI requires data. With 90% of the worlds data being generated in the last 2 years, this technology is quintessentially ‘exponential’. Data-driven healthcare brings with it new obstacles; not least digitising the archaic NHS documentation systems, along with the massive ethical hurdles of patient data-sharing and accountability. The teething process will prove tedious, and there will be those amongst us that refuse to submit to the technology, but if and when the evidence-base is sufficiently built, the ethical principle of non-malificence will give us no option but to fully embrace our digital future.

And judging by the success of DeepMind’s previous exploits, along with the look I saw in Dr. Keane’s eyes as we chatted, the pending evidence-base is a foregone conclusion.

Robert Lloyd


  2. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. 2015 518:529-33.
  3. Silver D, Schrittwieser J,  Simonyan K et al. Mastering the game of Go without human knowledge. 2017 Oct 18;550(7676):354-359.
  4. Burgess M. Google’s DeepMind wins historic Go contest 4-1.Wired 2016 Mar 15.
  5. How a computer beat the best chess player in the world – BBC
  6. Google AlphaGo computer beats professional at worlds most complex bard game go
  7. What is Machine Learning?
  8. Deep Learning – Wikipedia
  9. Public Lecture with Google DeepMind’s Demis Hassabis. Royal Television Society. Youtube.
  10. Google buys UK artificial intelligence start-up DeepMind. BBC.
  11. Elon Musk says he invested in DeepMind over ‘Terminator’ fears. The Guardian.
  12. Harnessing Deep Learning to unlock new insights from Ocular Health. Pearce Keane, lecture. Contact Innovatemedtec. Youtube.
  13. OCT rollout in every Specsavers announced. Optometry today.
  14. DeepMind: inside Google’s superbrain,Wired.
  15. Google’s DeepMind to analyse one million NHS eye records to detect signs of blindness. The Telegraph.
  16. DeepMind Health Research Partnership. Moorfields Eye Hospital website.
  17. Automated analysis of retinal imaging using machine learning techniques for computer vision. F1000 Research.
  18. The computer will assess you now. BMJ2016;355:i5680
  19. DeepMind and University College London Hospitals NHS Foundation Trust.
  20. DeepMind and the Royal Free.
  21. Alder Hey Children’s Hospital set to become UK’s first ‘cognitive’ hospital. Press release, 11 May 2016.
  22. Esteva A, Kuprel B, Novoa R et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118
  23. Doctor AI will see you now. Student BMJ.

Say Never!

say never


In my last blog I wrote about retained guidewires and why they are important to those of us in the Emergency Department. There were some tips on how to prevent retained guidewires through observership, redundancy, and good clear verbal and written documentation to promote absolute certainty that the guidewire has been removed. I also mentioned that this topic is one of patient safety, to the point that a retained guidewire in a patient is on the NHS England list of never events.

But, what are these ‘never events’? They exist, you’ve probably heard of the term at least, but do you know what they are? Do you know how they relate to your practice in the ED? These are questions that came up in the most recent FRCEM examination, but whilst they’re valuable knowledge for exams, they are also crucial to your day-to-day clinical work, and in trying to minimise harm to your patients.

Let’s find out a bit more.


What is a never event?

Never events have a very specific definition. They are a particular type of serious incident that meet all of the following criteria:

  • Potential to cause serious patient harm or death
  • Easily recognisable
  • Clearly defined
  • Wholly preventable
  • Evidence of previous occurrence
  • Risk of recurrence
  • National guidance for prevention in place


If a never event occurs, this denotes a failing in the system and should prompt a review of patient safety systems and procedures, ensuring implementation of any changes required to prevent recurrence. Reporting of such incidents is therefore necessary to improve patient safety, and to learn from our mistakes.

There is a list of never events which is published by NHS England and reviewed annually.


What’s on the list?

The list is divided into categories and then further into specific events.


  • Wrong site surgery
  • Wrong implant/prosthesis
  • Retained foreign object


  • Mis-selection of a strong potassium-containing solution
  • Wrong route administration
  • Overdose of insulin due to abbreviations or incorrect device
  • Overdose of methotrexate for non-cancer treatment
  • Mis-selection of high strength midazolam during conscious sedation

Mental Health

  • Failure to install collapsible shower/curtain rails


  • Falls from poorly restricted windows
  • Chest/neck entrapment in bedrails
  • ABO-incompatible transfusion or transplantation
  • Misplaced naso/orogastric tubes
  • Scalding of patients during washing/bathing


Which never events are relevant to the ED?

From looking at the above list you can immediately see a few events which are very relevant to the ED. ABO-incompatible transfusion, midazolam selection, and all of the other medication events (with the exception of methotrexate administration) might happen in your emergency department on a daily basis. However, there are other things we do in the ED which you might not think of straight away that also appear in this list.

We perform surgical interventions – chest drains, nerve blocks, or regional blocks, and it is important to make sure we are doing these in the correct site, otherwise we are performing wrong site surgery – a never event. It’s always crucial to look at the chest x-ray before you put a chest drain in, to ensure you don’t try to drain the healthy hemithorax. Note that the definition of wrong site surgery excludes blocks for pain relief, such as a fascia iliaca block (but that’s not an excuse to get the side wrong!), but does include procedural blocks e.g. to reduce dislocations or suture wounds. If a chest drain was being inserted in theatre, the patient would be marked, and the WHO checklist completed before the procedure. Do you do this in your ED?

As discussed in the last blog, retained foreign objects – guidewires – comprise half of the never events reported in EDs across the country.

If you work in one of those lucky emergency departments that has windows, then don’t forget if your patient falls out of a window then that too is a never event. If you have windows, you probably will have noticed that they open less than an inch, just to be sure.


Entrapment in bedrails is certainly something that has the potential to occur in the ED. We have a huge volume of elderly or confused patients, with poor mobility, who are unwell, with the potential to try to climb out of bed via any gap they can find.

Sometimes, we even insert feeding nasogastric tubes – think of the patient who comes in having pulled their tube out. We insert a new one and send them home. Make sure correct placement is assessed and documented before they leave or start feeding/medicating through it, to avoid misplaced tubes.

There’s only really one or two items on the list that don’t apply in the ED (and even those may do in rare cases, which is when we really need to be careful – when was the last time you prescribed methotrexate in the ED? With bed problems across the country, it may be that the patient you’re admitting needs their methotrexate whilst they’re still in your care…)


What do we do about them?

Go to your department. Take a look at each never event. What processes has your department got in place to prevent these occurring.

How do you ensure the correct strength of midazolam is selected? Who checks it? Do you know the correct strength of midazolam in your vials?

Do you have a procedural checklist? Do you use it? How do you ensure correct drain placement, correct block site, guidewire removal?

What is your reporting process? Don’t forget, if you don’t know what a never event is, how are you going to ensure they get reported?


So, how can you improve patient safety in your ED? Over to you.