Posted by 
Estimating the risk of patients dying is arguably one of the most difficult and stressful challenges physicians face. This has been especially true in the midst of the global COVID-19 pandemic, with doctors around the world repeatedly confronted with difficult decisions. In the best of cases, they have been able to adjust treatments and save lives. In the worst case scenario, however, physicians have to allocate scarce beds and life-saving machines in intensive care units. An international team led by researchers at the Max Planck Institute for Intelligent System has now developed an algorithm and trained it with machine learning methods to help medical professionals with mortality predictions. The algorithm can also be trained to predict mortality risk for other diseases, and thus support physicians in decision-making processes.
While hospital physicians collect a wealth of medical data on their patients, even specialists are often unable to predict whether an illness will lead to a person’s death until it is too late to save them. With COVID-19, for instance, advanced age and pre-existing conditions are relevant risk factors for serious disease, but by no means are they the only risks. Oxygen saturation, white blood cell count, and creatinine levels also play a role in health outcomes. “With these parameters, even experienced physicians cannot recognize clear patterns that would allow them to make predictions about mortality risk early enough to adjust treatment accordingly,” says Stefan Bauer, research group leader at the Max Planck Institute for intelligent Systems in Tübingen. By recognizing patterns in data, machine learning can provide valuable support.
An international team led by Stefan Bauer of the Max Planck Institute for Intelligent Systems and Patrick Schwab, formerly of Roche, thus developed an algorithm and trained it to predict individual mortality risk for patients with COVID-19 by drawing on the data of thousands of patients around the world. They call the algorithm Covews, which is short for “COVID-19 Early Warning System.” In addition to Max Planck scientists from Tübingen and Roche, researchers from Harvard University, Harvard Medical School, the Massachusetts Institute of Technology, Tübingen University Hospital, and Winterthur Cantonal hospital also contributed to the research project. Their paper “Real-time prediction of COVID-19-realted mortality risk using electronic health records” was published today in Nature Communications.
Predictions with 95 percent sensitivity and nearly 70 percent specificity
Drawing on medical data, Covews can reliably predict a patient’s risk of dying up to eight days in advance with a sensitivity of more than 95 percent. This means that in 95 out of 100 cases, the algorithm can detect whether a patient will die unless preventative measures are taken. At the same time, Covews works with a specificity of just under 70 percent for a prediction eight days in advance, meaning that in about 70 out of 100 cases in which death is predicted, the patients ultimately die. In other words, the algorithm sounds a false alarm in only 30 out of 100 cases and is significantly better for shorter time horizons. The algorithm can also be trained to make less sensitive, but more specific predictions. “But detecting all people with a high mortality risk, if possible, is more important than incorrectly predicting a high risk in some,” says Stefan Bauer. When the latter occurs, special therapeutic measures could be taken in more patients than necessary to avert presumed death.
To develop and especially to train Covews, the researchers used 33,000 anonymized data records from a cohort called Optum, which tracks patients in various hospitals in the United States. They fed the algorithm information about how several routinely collected patient health parameters evolved over the course of the disease, and whether or not the person died from COVID-19. As a result, Covews learned to identify patterns in the data sets that indicated a high risk of mortality. The international team then tested how accurately Covews estimated this risk on about 14,000 other data sets from the Optum cohort. “However, our algorithm not only predicts mortality risk with a high degree of certainty with data sets from this cohort, but also with data from other hospitals,” says Stefan Bauer. The researchers showed this by testing Covews on data from the TriNetX global health network, which includes about 5,000 patients with positive COVID tests in the U.S., Australia, India, and Malaysia. In these test cases in many different hospitals and regions around the world, Covews also predicted mortality risk very sensitively and specifically.
Treatment decisions must always remain in doctors’ hands
Although Covews makes reliable predictions, it will likely take quite some time before it is used in practice. “It often takes several years before such new techniques are used in everyday clinical practice,” says Stefan Bauer. This is partly because at many hospitals, the available data are not sufficiently structured, making the development of suitable software based on the algorithm particularly challenging. In any case, by making Covews freely available on the internet, the researchers are laying the groundwork for putting the algorithm into practice quickly. Not only could it be used for COVID-19 patients; with the right training, it could also predict mortality risk for other diseases.
Like most predictions using machine learning methods, Covews’ predictions are derived from correlations rather than causal relationships. Correlations can be purely statistical, meaning they are not causal. Bauer’s team also points out a limitation of Covews’ calculations: it is possible that the algorithm predicts treatment discontinuation rather than mortality. In that case, the predictions would not be based on medical facts alone.” Medical considerations are not the only factors that play a role in the decision to discontinue treatment,” says Stefan Bauer. Religious, cultural or personal attitudes can also lead people to stop undergoing treatment. For example, people may generally reject artificial respiration or refuse to accept life-saving measures out of fear of the long-term consequences of an illness. What’s more, family members or friends often have a say in such decisions. “Doctors must thus always decide on treatment measures,” says Stefan Bauer. “However, our algorithm can provide insights that people can’t derive from the data, and that can help with medical decisions.”
Max Planck Society