Enterprise AI Analysis

How Learners Engage with an LLM-Based Pedagogical Conversational Agent During Music Form Analysis

This study examines how university learners with different performance levels interact with a large language model (LLM)-based pedagogical conversational agent (PCA) during music analysis tasks. Using interaction logs from 41 undergraduate learners, we employed quantitative content analysis and Ordered Network Analysis (ONA) to investigate (a) learners' cognitive interaction strategies and (b) their perceptions of the PCA. The results reveal clear group-level differences in interaction patterns. High-performing learners (n = 24) exhibited more structured inquiry sequences, typically progressing from simple questions to explanatory requests and evaluative summarization. In contrast, low-performing learners (n = 17) relied more heavily on follow-up questions, prompt refinement, and verification moves. Despite these differences, learners across both groups reported positive perceptions of the PCA's usefulness and ease of use. These findings suggest that while LLM-based support facilitates access to analytical information, it does not automatically foster metacognitive or evaluative engagement, particularly for learners with lower prior competence. Accordingly, this study underscores the need for enhanced scaffolding mechanisms in LLM-based PCAs to better support low-performing learners in music analysis tasks by encouraging critical engagement with AI-generated explanations and promoting deeper problem-solving processes.

Authors: Lingxi Jin, Kyuwon Kim, Baicheng Lin, Mengze Hong, Hyo-Jeong So

Keywords: Large Language Models, Pedagogical Conversational Agent, Music Education, Ordered Network Analysis