An Optuna-Based Metaheuristic Optimization Framework for Biomedical Image Analysis

The success of Deep Learning (DL) in biomedical imaging heavily relies on optimal hyperparameter selection, which remains a complex and computationally intensive challenge. This paper introduces a metaheuristic-inspired Optuna framework for efficient hyperparameter optimization and validates its effectiveness using U-Net as a case study for brain MRI segmentation, leveraging Bayesian Optimization (BO) with adaptive pruning and Tree-structured Parzen Estimator (TPE). The proposed framework dynamically searches the hyperparameter space to maximize segmentation accuracy while reducing training overhead. The proposed framework adjusts key architectural, training, and regularization parameters, including the number of filters, optimizer, learning rate, and dropout rate, using a well-defined search space. Experimental results on a brain MRI dataset demonstrate that the proposed framework achieved mean scores of 0.941, 0.8763, and 0.983 for Dice Coefficient (DC), Intersection over Union (IoU), and Structural Similarity Index (SSIM), respectively, with 95% confidence intervals. These results confirm Optuna's effectiveness over traditional and metaheuristic baselines and demonstrate its scalability for broader biomedical AI applications.