Abstract
Introduction Grounded in self-regulated learning, we present a quasi-experimental design to examine how a generative artificial intelligence (AI) upgrade affects learning in an upper-level Environmental Data Science course. Methods We examine the educational effects of an AI upgrade (e.g., from ChatGPT-3.5 to ChatGPT-4o) by keeping the curriculum, pedagogy, and assessments constant across two cohorts (Spring 2024, n = 12 and Spring 2025, n = 13). Results The comparison suggests a significant gain in authentic, ill-structured project performance with no loss of AI-free conceptual and basic-skills exam performance, and points to a shift from peer-dependent to autonomous help seeking and increased positive sentiment toward AI support. Discussion Our central proposition is that effective pedagogical design is a necessary condition for AI upgrades to yield learning gains. Specifically, we advance the AI-affordance alignment framework where learning gains occur when AI capabilities are deliberately matched to task authenticity, the self-regulated learning phase, and pedagogical goals. The findings, while limited by small sample size, single-site design, and a focus on performance rather than direct measures of critical thinking, suggest that advances in AI can improve higher-order performance without eroding short-term conceptual recall, provided that effective pedagogical strategies for AI integration remain in place. These strategies include process transparency, scaffolded AI use, hybrid AI-resistant assessment checkpoints, metacognitive reflections, and active instructor mentoring. The study provides guidance for integrating current and future advances in AI into higher education, especially in research-based contexts where there is a need to balance innovation with the preservation of learner agency and academic integrity. Future research should directly investigate the impact of AI upgrades on student critical thinking and cognitive development.