The Learning Agency Lab

In the 2023 fiscal year, The Learning Agency Lab continued to build momentum in advancing educational technology through data science competitions and community engagement. By September, the Lab successfully launched three major competitions in the Open Data Assessment Fund (ODAF) series, which focused on improving educational outcomes through cutting-edge algorithm development.The first competition, The Quest for Quality Questions: Improving Reading Comprehension through Automated Question Generation, concluded in March 2023. This competition attracted 10 teams, with over 40 participants from around the world. The challenge addressed a critical issue in assessing student reading comprehension by developing models that automatically generate comprehension questions. These models, trained on a dataset of childrens storybooks, demonstrated the ability to produce human-like fluency in question generation, which presents a significant advancement in AI-supported education. By helping educators save time, these tools have the potential to broaden the range of texts students can engage with.In June 2023, the second competition, Predict Student Performance from Game Play, concluded with over 2,000 participants submitting 54,000 entries. Using data from the educational game Jo Wilder and the Capitol Case, this competition focused on advancing knowledge tracing to predict how students learn and perform in educational games. The top-scoring model provided accurate predictions of student success, surpassing human capabilities and paving the way for improved educational game development, targeted interventions, and real-time feedback for educators.The third competition, CommonLit - Evaluate Student Summaries, completed in October 2023. With 2,450 participants and nearly 45,000 entries, it focused on creating models that automatically assess student-written summaries for grades 3-12. The models, developed for both traditional accuracy and computational efficiency, make them suitable for low-resource environments. These models enable teachers to efficiently evaluate students summarization skills, encouraging more frequent use of this important instructional strategy by reducing grading burdens.The Lab also strengthened its engagement with stakeholders by establishing a Teacher Advisory Board, which provided critical feedback for refining competition ideas. The Board consisted of five teachers from a range of grades between K-12 and teach a variety of subjects. Several of the teachers work with low-income students or English Language Learners as well. Their interests, feedback, and ideas drove discussion and goals for many of the data science competitions this year and helped set the stage for next year. For instance, the overwhelming support for an AI detection tool led to a 2024 AI Writing Detection competition. The Lab has also forged strong relationships with vendors to maximize the real-world impact of its work. For example, the algorithms from the Jo Wilder competition were shared with one of the top educational gaming platforms in the U.S., demonstrating the Labs growing influence on the broader educational technology landscape. The Lab hopes to see the algorithm integrated into their future games to provide further insights during gameplay.In alignment with its mission to promote innovation and make data more accessible, the Lab launched the Learning Exchange platform in 2023. This platform hosts datasets and winning algorithms from completed competitions, making them available for public use and encouraging further research and development. Researchers have already begun utilizing the Exchange, with one standout example being a winning team from The Quest for Quality Questions competition whose paper was recognized at the 2023 BAE conference.Beyond its competitions, the Lab continues to foster the learning engineering community through its Learning Engineering Ambassador and Internship programs. The Ambassador Program ran a spring cohort, with one ambassador continuing in the fall. The program engaged students with a diverse range of educational interests, from K-12 learning to adult education. The ambassadors each hosted multiple events, demonstrating the broad scope and relevance of learning engineering. Additionally, the third cohort of the Learning Engineering Internship Program launched in spring 2023, providing undergraduate, graduate, and PhD students the opportunity to work with university labs and education-focused organizations. The interns contributed to a wide range of projects, from research to data visualization and coding, further advancing the Labs mission of empowering the next generation of learning engineers.Overall, the Labs efforts in 2023 significantly advanced AI and data science in education. These competitions and initiatives have produced valuable tools for educators and students alike, while also fostering a growing global community of learning engineers. With plans for future competitions and partnerships, the Lab is positioned for even greater impact in the coming year.