Integrating machine learning with statistical methods improves disease risk prediction models

Researchers at Peking University have conducted a comprehensive systematic review of integrating machine learning with statistical methods for disease risk prediction models, highlighting the potential of such integrated models in clinical diagnosis and screening practice. The study, led by Professor Feng Sun of the Department of Epidemiology and Biostatistics, School of Public Health, Peking University, was published in Health data science.

Predicting disease risk is crucial for early diagnosis and effective clinical decision making. However, traditional statistical models such as logistic regression and Cox proportional hazards regression often face limitations due to underlying assumptions that may not always apply in practice.

Meanwhile, despite their flexibility and ability to handle complex and unstructured data, machine learning methods have not consistently demonstrated superior performance over traditional models in certain scenarios. To address these challenges, integrating machine learning with traditional statistical methods could provide more robust and accurate predictive models.

The systematic review analyzed various integration strategies for classification and regression models, including majority voting, weighted voting, stacking, and model selection based on whether predictions from statistical methods and machine learning disagreed. The study found that integration models generally outperformed both statistical and machine learning methods when used alone. For example, stacking was particularly effective for models with over 100 predictors because it allows combining the strengths of different models while minimizing weaknesses.

“Our results suggest that integrating machine learning with traditional statistical methods can provide more accurate and generalizable models for disease risk prediction,” said Professor Feng Sun, the lead researcher. “This approach has the potential to improve clinical decision making and improve patient outcomes.”

Looking forward, the research team plans to further validate and improve existing integration methods and develop comprehensive tools to evaluate these models in various clinical settings. The ultimate goal is to establish more efficient and generalizable integration models tailored to different scenarios, ultimately advancing clinical diagnosis and screening practices.

Provided by Health Data Science