A new project will develop an AI assistant to help middle school students better understand scientific data - and provide tools for teachers to use in their classrooms.

The project, “Building AI Models to Help Middle School Students Interpret Science Diagrams,” aims to to develop, implement, and test a new AI assistant, the Representational Reasoning Assistant (RRA), to help middle school students interpret representations in their science classrooms. Representations such as diagrams, graphs, and charts are central to science and science education, explained Cindy Hmelo-Silver, Distinguished Professor and Associate Dean for Research and Development. However, learners often struggle with how to interpret science representations.

“This project will help middle school teachers provide individualized feedback to learners as they make sense of different kinds of visual representations such as graphs, charts, and diagrams,” said Hmelo-Silver. “This kind of individual feedback is challenging in regular-sized classes and these representations are important for students in addressing standards in science and math. We are excited to develop tools using state of the art AI models that use multimodal (i.e, language and visual information) to help support the work of middle school teachers.”

The project is in collaboration with Anne Ottenbreit-Leftwich, the Barbara B. Jacobs Chair in Education and Technology, and professor of Instructional Systems Technology at the IU School of Education, adjunct professor of Computer Science, and IU associate vice president for Learning Technologies, David Crandall, Luddy Professor of Computer Science at the Luddy School of Informatics, Computing, and Engineering, Joshua Danish, professor and department head of Curriculum and Instruction in the College of Education at the University of Illinois at Urbana-Champaign, and former professor and director of the Center for Research on Learning and Technology at IU, and Adriana Kovashka, associate professor and department chair of Computer Science at the University of Pittsburgh.

Throughout the three years of the project, learning sciences and computer science faculty will work with teachers and students from a large variety of schools, consistently gathering teacher feedback on initial models and versions of the AI assistant. The AI assistant interface will then be fine-tuned based on teachers and students' feedback as well as measurements of students' engagement and learning.