EVALUATING FINE-TUNED LLMS FOR MENTAL HEALTH SUPPORT: TEXT AND AUDIO MODALITIES

Engineering notes

Engineering notes will be added by the aipentium editorial team.

Chinese explanation / 中文解读

中文解读待补充：本站会优先为大语言模型、生成式AI、ChatGPT相关技术、计算机视觉、深度学习等高价值论文补充中文说明。

Original abstract

Study presents a comprehensive evaluation of a mental health language model, fine-tuned on a curated dataset of questions and conversational mental health data across text and preliminary audio modalities. The dataset comprises structured information from reputable mental health blogs, conversational data from therapist-patient interactions, and audio records. We leveraged the GPT-4 model as a reference against fine-tuned GPT-2, TinyLLaMA, DialoGPT and BART. Application of retrieval-augmented generation (RAG) pipeline and an audio-based emotion recognition module using HuBERT-Large-SUPERB-ER model to explore multimodal conditioning via inferred emotions (e.g. neutral, anger, happy, sad). Evaluation metrics included linguistic analysis (readability, sentiment, response diversity) and semantic similarity (ROUGE-L, BLEU, BERTScore), comparing the model’s responses with those from GPT-4. Our findings highlight the model’s capability to deliver accurate, empathetic, and user-centered mental health support, demonstrating its suitability for addressing a wide range of mental health inquiries. We plan to further improve the model's accuracy in this area.

Links and sources

Comments