Uncertainty-Aware Multimodal Fusion for Oral Lesion Classification

arXiv:2511.12268v3 Announce Type: replace-cross Abstract: Early detection of oral cancer and potentially malignant diseases is a major challenge in low-resource settings due to the scarcity of annotated data. We provide a unified approach for oral lesion classification that incorporates deep learning, spectral analysis, and demographic data. A pathologist verified subset of oral cavity images was curated from a publicly available dataset. Oral cavity pictures were processed using a fine tuned ConvNeXtv2 network for deep embeddings before being translated into the hyperspectral domain using a reconstruction algorithm. Haemoglobin sensitive, textural, and spectral descriptors were obtained from the reconstructed hyperspectral cubes and combined with demographic data. Multiple machine learning models were evaluated using patient specific validation. Finally, an incremental heuristic meta learner (IHML) was developed that merged calibrated base classifiers via probabilistic feature stacking and uncertainty-aware abstraction of multimodal representations with patient level smoothing. By decoupling evidence extraction from decision fusion, IHML stabilizes predictions in heterogeneous, small sample medical datasets. On an unseen test set, our proposed model achieved a macro F1 of 66.23% and an overall accuracy of 64.56%. The findings demonstrate that RGB to hyperspectral reconstruction and ensemble meta learning improve diagnostic robustness in real world oral lesion screening.

Related

• GlaBoost: A Multimodal Structured Framework for Glaucoma Risk Stratification
• Explaining Uncertainty in Multiple Sclerosis Cortical Lesion Segmentation Beyond Prediction Errors
• Missing-Modality-Aware Graph Neural Network for Cancer Classification
• Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis

Source: arXiv cs.CV | 2026-07-31