Challenges in the classification of medical pathologies in older adults: a review of the impact of imbalanced datasets in Machine Learning

Abstract

The development of Machine Learning (ML) models for the diagnosis and monitoring of medical pathologies in older adults faces multiple challenges, with one of the most critical being the issue of imbalanced datasets. In such cases, the scarcity of data corresponding to less frequent diseases affects the ability of algorithms to learn representative patterns, leading to models biased toward the majority class and a high rate of false negatives in critical conditions.

The objective of this study is to review the impact of class imbalance on the classification of medical pathologies in older adults and critically analyze current strategies to mitigate this issue. Traditional approaches such as oversampling (SMOTE, ADASYN), undersampling, and hybrid methods (SMOTE-ENN, SMOTE-Tomek Links) are compared, along with advanced Deep Learning techniques and synthetic data generation. Additionally, the study identifies more suitable evaluation metrics for assessing performance in imbalanced scenarios, such as AUC-PR, F1-score, and Balanced Accuracy.

The methodology follows a systematic literature review based on the PRISMA model, compiling relevant studies from high-impact scientific databases (PubMed, Scopus, IEEE Xplore). Throughout this process, specific inclusion and exclusion criteria were established to select research that specifically addresses the problem of class imbalance in medical applications for older adults.

Preliminary findings suggest that, while oversampling improves the representation of the minority class, it introduces the risk of overfitting and noise in the data. Undersampling, on the other hand, can compromise the information of the majority class, reducing the model’s overall predictive capacity. Hybrid techniques, according to the reviewed literature, appear to be a viable and superior alternative to the previous methods, as they combine the best aspects of both approaches and optimize class distribution within the dataset. However, they require precise calibration and careful evaluation in each specific application. Deep Learning models, such as Generative Adversarial Networks (GANs) and autoencoders, have been explored for synthetic data generation, showing potential in creating more realistic samples but with limitations in interpretability and high computational costs.

Given the above, although the hybrid approach presents relevant advantages, a definitive solution that completely eliminates the adverse effects of data imbalance in AI models applied to healthcare has yet to be identified. The issue remains open and represents an active area of interest within the scientific community, particularly at the doctoral research level, where new methodologies continue to be explored to enhance the fairness and reliability of predictive models in clinical settings. Therefore, it is crucial to continue advancing the development of more robust techniques that improve the detection of underrepresented pathologies in older adults, ensuring more accurate and effective diagnoses and medical care.