Measuring Educational Impact in the Age of GenAI: Why it Matters More Than Ever

The world of education is constantly ebbing and flowing between the pressures of change and the wisdom of tradition – the, “If it isn’t broke, don’t fix it” mantra. And yet, there are many problems that need solving and new heights of capability that we can reach under the awesome potential of new technologies like generative AI (GenAI).

Edtech is no longer a novelty—it’s a necessity. And with the emergence of GenAI, the potential for transformation is immense. But with great power comes great responsibility. As we embrace the possibilities of GenAI in education, we must prioritize measuring its impact and ensuring it truly delivers on its promise.

This is especially crucial when we consider what is at stake – the future of our planet under the stewardship of future generations. Delivering quality education to every child on the planet is going to take new approach and new ways of thinking. For small and large nations, independent schools and public schools alike, every investment into educational technologies needs to demonstrate tangible returns. We can no longer afford to implement technologies without a clear understanding of their effects on educational outcomes & engagement, student & teacher wellbeing, and operational efficiency.

The Importance of Evidence-Based Edtech

Edtech products developed with proven learning science principles are more likely to yield positive educational outcomes. As educators and policymakers become increasingly discerning, scientific rigor and research transparency are no longer optional—they are essential.

This shift is reflected in frameworks like the one developed by the International Center for Education Impact, which assesses edtech providers’ approaches to measuring their impact through evidence and research. TopSchool.ai is working with the Center through the DOHE EdTech Accelerator Program and we recently completed the ‘Research Orientation Analysis’ with their Co-Founder and Director, Natalia Kucirkova. This thought provoking exercise highlighted the importance of building measurement and analysis into everything we do. The promises of AI in Education are bold, but teachers are professionals, holding themselves to high standards of evidence based practice and this means we need to support our teachers in embracing AI but at the same time, ensure that it is effective in delivering on those promises. Sharing knowledge, measuring impact and being transparent helps us to support this need for research in GenAI in education.

TopSchool.ai’s Commitment to Impact

At TopSchool.ai, we firmly believe that impact is not just a buzzword—it’s the core of our mission. We are dedicated to developing AI-powered learning solutions that make a measurable difference in students’ lives. This means going beyond anecdotal evidence and employing rigorous evaluation methods to understand how our technology impacts:

Student Educational Outcomes: Are students demonstrating a deeper understanding of concepts? Are they achieving higher scores on standardized tests?

Student Wellbeing: Are students feeling more engaged and motivated in their learning? Are they developing a growth mindset and a love for learning?

Teacher Efficiency: Is our technology freeing up teachers’ time to focus on personalized instruction and student support? Is it helping teachers streamline administrative tasks and improve their overall workflow?

The GenAI Advantage: Amplifying Impact and Addressing Challenges

GenAI holds the potential to supercharge the impact of Edtech. However, recent studies, illustrate the need for careful design and implementation. While these studies demonstrate that GenAI can significantly improve educational performance, sometimes helping students learn twice as much in the same amount of time, they also reveal a potential downsides. Studies show that when access to GenAI is removed, students can perform worse than those who never had access. This suggests that simply allowing students to use off-the-shelf GenAI solutions can actually hinder their learning. This also further strengthens the case for careful measurement of impact during this transformational period of AI in education.

The research emphasizes that successful AI-based tutoring systems need to be designed with pedagogical best practices in mind. Key elements include:

Active Learning Facilitation: Encouraging students to actively engage with the material rather than passively consuming information.

Cognitive Load Management: Presenting information in a way that avoids overwhelming students and facilitates understanding.

Growth Mindset Promotion: Fostering a belief in students’ ability to learn and improve.

Content Scaffolding: Breaking down complex problems into manageable steps and providing appropriate support.

Accuracy Assurance: Ensuring the AI provides accurate information and avoids “hallucinations” or factual errors.

Personalized Feedback: Tailoring feedback to each student’s individual needs and mistakes.

Self-Pacing: Allowing students to learn at their own pace and revisit concepts as needed.

Beyond the Hype: Measuring What Matters

As we navigate the exciting possibilities of GenAI, let’s not lose sight of the bigger picture. The true value of any Edtech solution lies in its demonstrable impact on student learning and well-being. At TopSchool.ai, we are committed to developing AI-powered tools that not only enhance learning but also empower students and teachers alike. And we will continue to rigorously measure our impact to ensure that our technology is making a real difference in the lives of those we serve.

By Joseph Telfer, CEO and founder of TopSchool.ai

Leave a Comment

Your email address will not be published. Required fields are marked *