New Framework for Long-tailed Recognition in Imbalanced Multi-modal Data

Researchers Heegeon Yoon and Heeyoung Kim have introduced a novel deep learning framework designed to address long-tailed recognition challenges within highly imbalanced multi-modal datasets. Traditional models often exhibit bias toward majority classes and are limited to single-modal inputs, failing to leverage complementary data sources. This new approach extends multi-expert architectures to handle heterogeneous inputs, such as images and tabular data, by fusing them into a unified representation. The system utilizes modality-specific networks to estimate the informativeness of each data source, applying confidence-guided weights to dynamically modulate the fusion process. This ensures that more informative modalities significantly influence the final decision. Specialized training and testing procedures were developed to accommodate diverse modality combinations. Extensive experiments on both benchmark and real-world datasets demonstrate that this method effectively integrates multi-modal information and outperforms existing techniques in handling class-imbalanced scenarios, showcasing superior robustness and generalization capabilities for complex AI applications.