This research investigates the impact of Large Language Models (LLMs), such as ChatGPT, on developing academic critique skills among Computer Science undergraduates taking a fourth-year Ubiquitous Computing course. Each week in the course, during the fall of 2023, students will be tasked with critiquing two academic papers. Initially, critiques are authored without any assistance from ChatGPT and assessed, forming a baseline for comparison. Subsequently, students interact with ChatGPT to enhance their critiques, and these revised versions are saved and assessed. The study aims to evaluate and compare the differences between student-authored critiques and those co-authored with ChatGPT's assistance. The methodology incorporates an iterative process of independent writing and AI-assisted revision, offering a comprehensive examination of how LLMs contribute to improving academic writing and critiquing skills. Evaluations are conducted objectively using widely accepted tools such as the System Usability Scale and readability metrics like the Flesch Reading Ease Score, Gunning Fog Index, among others. The research promises to shed light on the potential role of LLMs in enhancing students' critical thinking and writing abilities, providing valuable insights into integrating AI tools within educational practices. It also explores the transformative potential of AI-driven tools in supporting student learning and fostering a more personalized and engaging process of academic critique. Furthermore, the findings of this research have broad implications for the learning sciences, pedagogy, and educational technology. They provide actionable insights into how AI can be harnessed to augment traditional learning methods, foster critical thinking, and personalize education.

Authors:
Ed Sykes, Sheridan College, Canada
Roland Van Oostveen, Ontario Tech University, Canada
Wendy Barber, Ontario Tech University, Canada

About the Presenter(s)
Dr Ed Sykes is a University Professor/Principal Lecturer at Sheridan College, McMaster University, University of Toronto in Canada

See this presentation on the full schedule – Saturday Schedule

Bookmark