Identification of suicidal presentations to ED using a machine learning algorithm

Overview

Initiative type

Service Improvement

Status

Deliver

Published

08 February 2020

Summary

To overcome the limitations inherent to the recording of suicidal presentations to Emergency Departments (EDs), Gold Coast Mental Health Services developed a machine learning algorithm - Searching EDIS for Records of Suicidal Presentations (SERoSP) - that identifies relevant presentations with a high degree of accuracy.

The software program was developed to identify suicidal and self-harm presentations through the use of machine learning. It is a crucial component in the two-step methodology considered gold standard for suicide prevention (accurate data and manual review of cases). It is also a time- and cost-effective solution to enable a reliable identification of suicidal presentations to GCHHS’s EDs. This can inform on the resources required for the provision of optimal care for consumers in suicidal crisis and assist in the evaluation of the Suicide Prevention Strategy.

The tool was developed in collaboration with Bond University and Queensland Health's Healthcare Improvement Unit. The project was nominated as a finalist in the 2019 Bond University Sustainable Healthcare Award.

Key dates

Apr 2019 - Jun 2020

Partnerships

Bond University and Queensland Health's Healthcare Improvement Unit

Aim

The aim of this initiative was to develop an innovative, time- and cost-effective solution to enable a reliable identification of suicidal presentations to GCHHS's EDs. This can inform on the resources required for the provision of optimal care for consumers in suicidal crisis and assist in the evaluation of the Suicide Prevention Strategy.

Benefits

When translated into monetary value, the dramatic reduction in the need for human labor in identifying and coding suicidal and self-harm presentations to EDs with the help of SERoSP represents substantial savings for our health services. In fact, in an environment that requires ever more stringent monitoring of expenditures, a sustained review of a large number of ED presentations would likely not be possible in the absence of this machine learning tool. Furthermore, this work is ongoing, which means that the demonstrated savings accumulate over time.

Background

It has recently been shown that suicide-related presentations are grossly under-enumerated in ED administrative data (Sveticic et al., 2019).

Solutions implemented

In February 2018, Prof Chris Stapelberg created a software program for identification of suicidal and self-harm presentations through the use of machine learning, titled Searching EDIS for Records of Suicidal Presentations (SERoSP).

The SERoSP program is written in MATLAB, and employs 136 variables recorded in EDIS, each assigned a unique weight, optimised by an evolutionary algorithm, denoting their contribution to a final score. SERoSP was shown to have a 95% sensitivity and 92% specificity in identifying suicidal and self-harm presentations (Stapelberg et al., 2018).

The use of SERoSP represents a crucial component of the two-step methodology considered as a 'gold standard'TM by the Suicide Prevention Team at GC MHSS (described in more detail in paper by Sveticic et al., 2019):

  1. The machine learning algorithm is applied to the entire collection of data in EDIS (prior to April 2019) or FirstNet (post April 2019), extracting relevant suicidal or self-harm presentations with a high degree of sensitivity and specificity.
  2. Manual review of the cases identified through SERoSP is undertaken by trained research officers to categorise them into one of the following groups: not applicable; suicidal ideation; non-suicidal self-injury; suicide attempt; or ambiguous.

Evaluation and results

The use of SERoSP tool has been able to dramatically reduce the time requiring human labor in determining the volume and types of suicidal or self-harm presentations to EDs. The time needed to identify and code ED presentations is reduced by around 30-times with the help of SERoSP. For example, we estimated that coding of 6-months' worth of data using only human labour would require 8,420 person-hours, and the use of SERoSP reduces required person-hours to 269 hours.

Prof Stapelberg and his team are currently in the process of publishing a journal article outlining the methodology behind the development of SERoSP.

Lessons learnt

The large amount of suicidal presentations to EDs, not only on the Gold Coast but Australia-wide, substantiates the need for utilisation of innovative methods through standardized and automated tools, capable of mining complex data. The development of SERoSP has demonstrated the utility of machine learning applications in achieving this.

The greatest challenge with the use of SERoSP has been the transition from the use of EDIS to FirtstNet as the electronic medical record system in Eds. At GCHHS, this transition occurred in April 2019, and we are currently in the process of adapting SERoSP to continue with data extraction in this new environment.

Key contact

Jerneja Sveticic

Research Officer

Gold Coast Hospital and Health Service

Email:  jerneja.svetici@health.qld.gov.au

Last updated: February 2020

Back to top