LLM as Tutor: AI-Guided Learning design for LLM

Three questions to the project team

What was the particular challenge of the project from a UX point of view?
The biggest challenge was redefining the objective: how do you preserve the strengths of an LLM—its speed, intelligence, and ability to explain complex concepts—while encouraging students to think independently? The UX challenge was guiding students without giving them the answer in a way that promoted learning without causing frustration. Every interaction had to strike a careful balance between support and discovery, revealing just enough to build understanding while maintaining confidence and momentum. Designing that balance consistently across six cognitive levels and four tool ecosystems was the most complex part of the project.

What was your personal highlight in the development process? Was there an aha!-moment, was there a low point?
The aha moment came when we found a common pattern across the heatmaps generated from our rubric-based evaluation of all four LLMs: performance dropped as cognitive demand rose, with the widest gaps in Pedagogical Quality and Interaction, not accuracy. That reframed everything. This was a UX problem, not a model problem. The low point came when ChatGPT removed Study Mode mid-study, breaking our test plan. We turned the disruption into our pivot, positioning ChatGPT as the redesign target rather than a benchmarked tool. Looking back, the constraint sharpened the work. It forced us to commit to one interface and design with conviction rather than hedge across four.

Where do you see yourself and the project in the next five years?
While the prototype is valuable today, we believe the project's lasting contribution will be the evaluation rubrics we created. AI tools are evolving rapidly, but the rubric is intentionally tool-agnostic, allowing it to assess new systems as they emerge while continuing to ask the same fundamental question: Is this AI tool genuinely helping students think and learn? Looking ahead, we hope to open-source the framework so educators and researchers can adapt and apply it across future tools and learning environments. As for us, we will continue working on designing interfaces that support how people actually learn rather than simply optimizing for answer delivery.