Enterprise AI Analysis
Integrating GenAI into HCI Pedagogy: Supporting Students' Learning of Evaluation Methods
Abeer Aziz, Ahmed Kharrufa, and Ian Johnson from Newcastle University
While recent focus on generative AI (GenAI) in education has centred on ideation and content creation, its role in supporting methodological learning remains underexplored. This study explores how GenAI can help undergraduate HCI students understand evaluation methods. We incorporated GenAI into group activities by enabling students to compare GenAI's suggestions with their own evaluation ideas and reflect on the differences. Additionally, we used GenAI as a facilitator throughout the steps of user evaluation methods by providing comprehensive and detailed prompts tailored for use in their preferred tool. Results show that GenAI helps students understand evaluation concepts, compare methods, and clarify differences, thereby fostering reflection on their choices. However, concerns about oversimplification, trust, and reduced critical thinking were noted. We suggest that, with proper scaffolding, GenAI can serve as a valuable supplementary teaching tool rather than a replacement, offering new insights into its responsible use in HCI education.
Executive Impact & Key Findings
This study highlights GenAI's significant role in enhancing methodological learning, particularly in HCI evaluation methods, alongside identified areas for critical human oversight.
0%
Students Found GenAI Helpful
0%
Heuristic Evaluation for GenAI
0+
Hours Reclaimed (Projected Annually)
0%
Increase in Very Helpful Perception
Deep Analysis & Enterprise Applications
Select a topic to dive deeper, then explore the specific findings from the research, rebuilt as interactive, enterprise-focused modules.
Reflective Comparison
Structured Guidance
Documentation Aid
Limitations & Oversight
GenAI as a Reflective Comparison Partner
GenAI served as a valuable partner in prompting students to compare their independent evaluation ideas with AI-generated suggestions. This process helped surface considerations students had not thought of, such as cognitive load and context of use, encouraging deeper and more critical thinking about the scope of evaluation.
However, students also identified important gaps in AI-generated suggestions, particularly regarding contextual and participant-specific factors (e.g., testing within subjects, natural environment of a classroom). This dynamic fosters critical reflection rather than passive acceptance.
GenAI for Structured Guidance & Scaffolding
Students primarily valued GenAI's role as a structured guide that supported focus, consistency, and completeness. It acted "like a structured guide that kept me focused on each step... successfully playing the role of a facilitator and walking me throughout the evaluation in a perfect order."
This was particularly beneficial for novices, helping them "understand what a cognitive walkthrough really is" and making complex tasks more accessible. It reduced cognitive load and saved time by organizing the process efficiently.
GenAI as a Documentation Aid
GenAI facilitated systematic documentation of evaluation findings. Its structured prompts helped students stay organized, ensuring consistent recording of key evaluation elements.
Students appreciated that it "ensured that I record success/failure points and design suggestions well and that my evaluation was well organized and consistent." The interaction pacing, where GenAI "waited until I completed the first step," further aided concentration.
GenAI's Limitations & Human Oversight
Despite benefits, students raised concerns about GenAI's lack of human intuition, emotional understanding, and contextual awareness. It "couldn't fully capture real user struggles or visual issues" and often felt "too robotic" or repetitive.
The usefulness of AI output was heavily dependent on prompt quality, and over-reliance was cautioned against, as it could "limit independent exploration and critical thinking." This underscores the need for human judgment and ethical considerations.
Cognitive Walkthrough Process with GenAI
Find 2-3 books by criteria
→
Get APA citation for each
→
Add both books to Favourites
→
Check Favourites list
| Aspect | GenAI Strengths | GenAI Limitations |
|---|---|---|
| Pedagogical Support |
|
|
| Reflection & Critical Thinking |
|
|
| Contextual Understanding |
|
|
| Methodological Adherence |
|
|
Projected ROI for AI-Enhanced Learning
Estimate the potential savings and reclaimed hours by integrating AI as a pedagogical scaffold within your organization's learning frameworks, based on similar efficiencies observed.
Calculate Your Potential Impact
Projected Annual Savings
$0
Reclaimed Instructor/TA Hours
0
Your GenAI Integration Roadmap for HCI Education
Based on the study's findings, here's a strategic roadmap for effectively integrating GenAI into your HCI pedagogy, ensuring a balanced approach that maximizes benefits while mitigating risks.
Phase 1: Initial Integration & Comparative Learning
Introduce GenAI as a reflective comparison partner, allowing students to generate their own evaluation ideas first, then compare them with AI suggestions. Facilitate group discussions on discrepancies to foster critical thinking and contextual awareness.
Phase 2: Structured Scaffolding & Guided Practice
Utilize GenAI as a procedural guide for complex evaluation methods like Cognitive Walkthroughs. Employ detailed prompts to scaffold multi-step tasks, ensuring systematic completion and reduced cognitive load for novices.
Phase 3: Fostering Critical Reflection & Instructor Oversight
Emphasize GenAI's limitations regarding human intuition and contextual understanding. Instructors should focus on higher-level feedback, ethical considerations, and promoting student autonomy to question AI outputs.
Phase 4: Longitudinal Study & Refinement
Conduct ongoing assessments to measure the long-term impact on students' evaluation literacy and responsible AI usage. Adapt pedagogical strategies based on feedback and evolving AI capabilities.
Ready to Transform Your HCI Pedagogy?
Leverage the power of GenAI to enhance student learning of evaluation methods. Schedule a personalized consultation with our experts to design a tailored integration strategy for your institution.
Ready to Get Started?
Book Your Free Consultation.
Let's Discuss Your AI Strategy!
Lets Discuss Your Needs
Select a Date
Sun
Mon
Tue
Wed
Thu
Fri
Sat