Skip to main content

Milestone 1: Learning Gradient Descent with and without an AI Scaffold

[!NOTE] Overleaf Project: git.overleaf.com/6ac144780808c76eacfe6f31
Last Synced: 2026-10-08 13:16:23 (via PA Git Sync)
Source File: main.tex

Learning Gradient Descent with and without an AI Scaffold

Authors: Team Kyoto baby: Isna, Karo, Stergios

Background: Why Gradient Descent?

Gradient descent is the optimization routine behind almost all of modern machine learning. Our team started from the idea of teaching novices to program neural networks. During our discussions we concluded that end-to-end implementation is too demanding for novice participants: code, architecture, and optimization would be learned at once, and the study could not tell which part caused learning. We therefore narrowed the project to one core concept, and compared three candidates (gradient descent, text analytics / dimensionality reduction, recursion). We chose gradient descent because:

  • It keeps the neural-network umbrella. It is the key mechanism of neural-network training and complements the introductory material our participants see in their machine learning course.
  • It is self-contained and testable in one hour. It has clear right and wrong answers (compute a step, predict a trajectory, diagnose a failing run), and its typical misconceptions are well known: a larger learning rate is always faster, the algorithm always finds the global minimum, and the update goes along the gradient.
  • A digital tool can add value beyond high-school calculus. Participants know derivatives, but the behavior of the algorithm (step size, curvature, oscillation in ravines, stalling at saddles) is hard to grasp from equations alone and becomes visible in an interactive 3D landscape.

Target population: Bachelor students (about 20), preferably second-year, who know derivatives and partial derivatives but have not yet been taught gradient descent. Master's students are excluded to keep the baseline comparable.

Theoretical grounding: We build on the course principle that learning depends on the cognitive activity learners perform with the technology, not on the technology itself, and on the ICAP framework. A visualization alone supports passive or active engagement, depending on how learners use it. The AI affordance is meant to push learners towards constructive and interactive engagement by prompting them to predict, explain, and revise. Our question is therefore not "does AI help?" but "does this AI affordance change what learners do, and does that improve learning?"

Learning Goal

Overall learning goal: Learners understand gradient descent well enough to apply it to a simple function and to reason about how its behavior depends on the learning rate and on the shape of the loss surface.

Learning objectives: After the session, learners can:

  • Compute one gradient descent update for a given one- or two-variable function, starting point, and learning rate, with the correct numerical result.
  • Explain in one or two sentences why the update moves opposite to the gradient.
  • Predict whether a run with a given learning rate will converge slowly, converge quickly, oscillate, or diverge, and justify the prediction.
  • Diagnose a given loss curve or trajectory that fails to converge, and propose a change of learning rate that fixes it.

Objectives 1–2 target knowledge and understanding, while 3–4 target application and diagnosis.