Beyond Polarity: Continuous Affect-Enhanced Multimodal Aspect-Based Sentiment Classification

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Multimodal aspect-based sentiment classification (MABSC) requires aspect-level sentiment inference from textual–image data that jointly convey opinions. Yet most existing approaches primarily exploit discrete polarity patterns and generic visual embeddings, making them less effective when the affect is subtle, implicit, or expressed through imagery. In this work, we propose <b><i>VADE</i></b>, a Valence–Arousal–Dominance (<b><i>VAD</i></b>)-<b><i>E</i></b>nhanced MABSC framework that brings continuous VAD signals into multimodal sentiment reasoning and learns emotion-sensitive image representations. Specifically, we design a <b>VAD encoder</b> to extract continuous affect cues from text for aspect-level sentiment reasoning. Furthermore, we fine-tune a CLIP-based image encoder on affect-enriched image–text pairs to obtain visual representations that are more sensitive to sentiment cues. To support the fine-tuning process, we construct an affect-enriched image–text dataset, <b><i>Senti-COCO</i></b>, by rewriting MSCOCO captions with a multimodal large language model, yielding large-scale image–text pairs with richer affective expressions. Experiments on two mainstream datasets, Twitter‑15 and Twitter‑17, show that <b>VADE</b> achieves state-of-the-art performance, demonstrating the effectiveness of incorporating Valence, Arousal, and Dominance (VAD) signals for MABSC.

Links and sources

Comments