Advancing the Science of Teaching with Tutoring Data: A Collaborative Workshop with the National Tutoring Observatory

This full-day and in-person workshop aims to bring together members of the ACM Learning @ Scale, Educational Data Mining, and AI in Education communities to share progress, identify common challenges, and explore collaborative solutions to understanding effective teaching and tutoring moves. The structure of this workshop will include presentations of accepted papers (see our Call for Papers below), a demonstration by the National Tutoring Observatory (NTO), and a panel discussion featuring key researchers in tutoring and teaching. 

The National Tutoring Observatory (NTO) is leading the creation of the Million Tutor Moves dataset—the largest open-access collection of tutoring interactions—using generative AI to accelerate the science of teaching at scale. This workshop will unite the Learning at Scale community to share progress, tackle shared challenges, and explore joint solutions. The agenda includes paper presentations, interactive demos, and a panel with researchers, developers, and tutoring providers, all aimed at advancing a shared vision for impactful, data-driven tutoring through collaborative research.

The target audience for this workshop includes researchers, educators, EdTech developers, policy makers, and leaders of tutoring organizations interested in tutoring and teaching research and AI-assisted learning. This workshop will be particularly relevant to members of the Learning at Scale community who are working at the intersection of AI and human tutoring to create individualized and effective learning experiences. Participation requirements include a device to access accepted papers, demonstrations, and materials, and an interest in advancing research on the science of learning and teaching. 

Workshop Format

This will be a full-day, in-person workshop with the following:

1. presentations of accepted papers with Q&A

2. a demonstration by the NTO

3. a moderated panel with audience participation focused on upcoming research in this area

4. a closing summary and discussion session

These activities will be oriented toward the workshop goals to develop a shared understanding of the current state of tutoring and teaching data and to pose key research questions and challenges for future research. The length of presentations will be determined by the organizing committee based on the maturity of the work, level of interest among L@S, and significance to the themes. A demonstration will be presented by the NTO. A subsequent whole-group discussion with moderated panel will contain a question-and-answer period with in-person panelists. A summary of the key issues and responses from the panel discussion, along with commonalities among accepted papers, will be published in Volume 2 of the conference proceedings.

Submission Guidelines

We will solicit papers relevant to the themes using the research paper format described in the conference call for papers guidelines. Papers will go through a double-blind review process. Please omit author names and affiliations in your submission.

Reviewers will be required to make a recommendation of either acceptance or rejection of the paper and explain their reasoning behind their decision. They will assess the paper based on three criteria, using a scoring system of -1, 0, or 1; alignment with the workshop’s theme, level of interest to L@S, and overall quality. Authors of accepted papers will provide presentations at the conference (more details to be announced).

Accepted papers are non-archival to encourage submissions of exciting and important work.

Paper reviewing and notification to authors will be handled using the workshop EasyChair account (link above). Submissions must be in PDF format and anonymized for double-blind review. All papers must follow the ACM 2-column proceddings template (Word or Latex), written in English, contain original work, and not be under review for any other venue while under review for this conference.

Submitted papers should be between 3 - 10 pages in length, excluding references.

If you have any questions regarding the submission process, reach out to:

• Danielle Thomas (drthomas@cmu.edu)

• Rene Kizilcec (kizilcec@cmu.edu)

List of Topics

This workshop aims to facilitate discussion and engagement among the Learning at Scale community. In particular, the workshop hosts updates on progress, findings, and challenges to collecting, pre-processing, annotation, and modeling interactional datasets. The goal of this workshop is to foster discourse, exchange valuable insights, make connections with the community, and develop a potential user base of contributors and users of the Million Tutor Moves repository. We invite empirical and theoretical papers aligned with the listed themes, particularly (but not exclusively) within the following areas of research and application:

• Understanding tutoring and teaching moves: Investigating in-the-moment instructional strategies, tutor-student interactions, and contextual factors that enhance learning outcomes.

• Data pre-processing, standardization, and deidentification: Developing scalable methods for cleaning, structuring, and anonymizing large-scale tutoring data, ensuring interoperability, privacy compliance, and reproducibility in educational research. 

• Multimodal data collection and annotation: Addressing challenges in capturing, segmenting, and annotating large-scale tutoring interactions across text, audio, and video.

• Predictive modeling of interactional data to learning outcomes: Exploring how human and AI-driven tutoring models influence student performance, engagement, and short- and long-term learning outcomes using machine learning and learning analytics

• Data sharing, privacy, and legal frameworks:  Examining ethical, legal, and policy considerations for securely sharing and protecting tutoring data while enabling open research.

• Fairness, equity, and inclusion: Developing inclusive approaches, mitigating bias, and ensuring equitable access within large-scale tutoring data to promote fairness in educational  AI applications.

• Key Challenges: Identifying barriers, considerations, and challenges to creating a collaborative infrastructure to share data analysis workflows and analysis routines

• Interoperability and scalable data management: Designing collaborative, shareable data management systems that integrate with existing educational technologies and platforms. 

Organizing committee (alphabetical order)

Dorottya Demszky, Stanford University; ddemszky@stanford.edu

René F. Kizilcec, Cornell University; kizilcec@cornell.edu

Kenneth R. Koedinger, Carnegie Mellon University; koedinger@cmu.edu

Josh Marland, Cornell University; jmarland@cornell.edu

Doug Pietrzak, FreshCognate; dougpietrzak@gmail.com

Justin Reich, Massachusetts Institute of Technology; jreich@mit.edu

Rachel Slama, RAND; rslama@rand.org

Danielle R. Thomas, Carnegie Mellon University; drthomas@cmu.edu

Amalia Toutziardi, Massachusetts Institute of Technology; amaliat@mit.edu

Venue

Date and time:  

July 21, 2025, 9 am - 4 pm (Central European Time)

Location: 

Room TBD, University of Palermo, Piazza Marina, 61, 90133, Palermo, Sicily

Contact

For questions about the L@S workshop, contact: 

Danielle R. Thomas: drthomas@cmu.edu 

Rene Kizilcec: kizilcec@cornell.edu