MVQAM:exploring state space models for medical visual question answer Mamba
摘要
Medical visual question answering(MedVQA)is a complex multimodal task.In this domain,multimodal fusion models that leverage the attention mechanism and transformer architecture have been extensively utilized.However,attention-based fusion models face limitations due to constraints in training datasets and global receptive fields,which hinder their ability to incorporate additional information effectively.Furthermore,the self-attention mechanism inherent in Transformer models presents challenges for real-time applications because of its high computational cost resulting from quadratic complexity.Recent studies indicate that state space models(SSM),exemplified by Mam-ba,can effectively model long-range interactions while preserving linear computational complexity.To address the limitations mentioned above,this paper proposes a novel MedVQA framework based on state-space modeling,termed MVQAM.As the core component of MVQAM,the Mamba vision correlation connector(MVCC)module adopts a dual-branch hybrid structure that synergizes convo-lutional neural networks(CNN)and Mamba.Specifically,CNN branch extracts local fine-grained features from medical images through stacked convolutional layers,while the Mamba branch captures long-range contextual dependencies in the feature map.The two branches'outputs are aggregated via linear projection to generate comprehensive visual feature representations.This integration not only enhances the efficiency of feature extraction but also reduces the overall computational complex-ity of the framework by leveraging Mamba's linear complexity advantage.Experimental results dem-onstrate that MVQAM surpasses existing multimodal fusion models on both SLAKE-EN and visual question answering in radiology(VQA-RAD)datasets in terms of accuracy,convergence speed,and parameter efficiency.These findings emphasize the significant potential of SSM in advancing MedVQA tasks.
年,卷(期):
2026, 32(3)
机标分类号:
TP181
TP3
TP273
在线出版日期:
2026年9月28日