A research design for an intervention study to measure the impact of GAI chatbots on student learning through simulated cooperative learning.

Research output: Contribution to conferenceConference abstract for conferenceResearchpeer-review


Peer feedback and cooperative learning are concepts that are often attributed with having an impact on student performance, and therefore a focus in the development of educational technology. The integration of Generative AI into higher education practices raises questions about its potential role in enhancing student learning.
This submission discusses the use of LLM-based chatbots in educational settings, emphasizing the technology’s unique ability to simulate interactions and conversation. The proposed study examines how chatbots can support learning through dialogical processes, thereby facilitating self-paced reflection and engagement with subject matter, that are traditionally assisted by peers. The proposed study aims to determine if students regularly using AI chatbots for reflection and feedback achieve comparable learning outcomes to those engaging in peer discussions, and potentially better outcomes than students practicing solo self-reflection. The following research question is therefore proposed: How can AI chatbots support student learning through dialogical processes?
A quasi-experimental intervention study will be conducted with four groups: one using a chatbot, another encouraged to do reflections and feedback with peers, a third group focused on self-reflection and self-initiated feedback, and a control group without intervention.
The study will use pre and post-test surveys to measure cognitive load, emotion, and immersion, as ways of evaluating student learning and retention.
This research design builds upon a pilot study indicating that AI coaching significantly affects the style and characteristics of written feedback.
The study is proposed as the main part of a future Ph.D. project, which is currently in the preparatory phase, that will also include a qualitative study based on observations and interviews, as well as a comparative Natural Language Processing analysis of feedback comments, to explore the broader impacts of AI chatbots on teaching and learning processes, this roundtable submission invites discussions and suggestions concerning both method and theory.
Original languageEnglish
Publication date21 Mar 2024
Number of pages3
Publication statusAccepted/In press - 21 Mar 2024
EventJURE 2024 - Faculty of Psychology of the University of Sevilla, C. Camilo José Cela, s/n, 41018 Sevilla., Sevilla, Spain
Duration: 24 Jun 202428 Jun 2024
Conference number: 28th


ConferenceJURE 2024
LocationFaculty of Psychology of the University of Sevilla, C. Camilo José Cela, s/n, 41018 Sevilla.
Internet address

Cite this