\textbf{Learning Gradient Descent with and without an AI Scaffold
[!NOTE] Overleaf Project: git.overleaf.com/6ac144780808c76eacfe6f31
Last Synced: 2026-10-08 13:35:21 (via PA Git Sync)
Source File:main.tex
\textbf{Learning Gradient Descent with and without an AI Scaffold
Background: Why Gradient Descent?
Gradient descent is the optimization routine behind almost all of modern machine learning. Our team started from the idea of teaching novices to program neural networks. During our discussions we concluded that end-to-end implementation is too demanding for novice participants: code, architecture, and optimization would be learned at once, and the study could not tell which part caused learning. We therefore narrowed the project to one core concept, and compared three candidates (gradient descent, text analytics / dimensionality reduction, recursion). We chose gradient descent because:
* **It keeps the neural-network umbrella.** It is the key mechanism of neural-network training and complements the introductory material our participants see in their machine learning course.
* **It is self-contained and testable in one hour.** It has clear right and wrong answers (compute a step, predict a trajectory, diagnose a failing run), and its typical misconceptions are well known: a larger learning rate is always faster, the algorithm always finds the global minimum, and the update goes *along* the gradient.
* **A digital tool can add value beyond high-school calculus.** Participants know derivatives, but the behavior of the algorithm (step size, curvature, oscillation in ravines, stalling at saddles) is hard to grasp from equations alone and becomes visible in an interactive 3D landscape.
Target population.: Bachelor students (about 20), preferably second-year, who know derivatives and partial derivatives but have not yet been taught gradient descent. Master's students are excluded to keep the baseline comparable.
Theoretical grounding.: We build on the course principle that learning depends on the cognitive activity learners perform with the technology, not on the technology itself, and on the ICAP framework. A visualization alone supports passive or active engagement, depending on how learners use it. The AI affordance is meant to push learners towards constructive and interactive engagement by prompting them to predict, explain, and revise. Our question is therefore not does AI help?'' but does this AI affordance change what learners do, and does that improve learning?''
Learning Goal
Overall learning goal.: Learners understand gradient descent well enough to apply it to a simple function and to reason about how its behavior depends on the learning rate and on the shape of the loss surface.
Learning objectives.: After the session, learners can:
* **Compute** one gradient descent update for a given one- or two-variable function, starting point, and learning rate, with the correct numerical result.
* **Explain** in one or two sentences why the update moves opposite to the gradient.
* **Predict** whether a run with a given learning rate will converge slowly, converge quickly, oscillate, or diverge, and justify the prediction.
* **Diagnose** a given loss curve or trajectory that fails to converge, and propose a change of learning rate that fixes it.
Objectives 1--2 target knowledge and understanding, objectives 3--4 application and analysis (Bloom). Each uses an action verb and can be measured with a test item.
Scope.: To fit the one-hour limit, the study uses plain gradient descent with the learning rate as the main parameter. Optimizer variants (Momentum, RMSprop, Adam), which the prototype also offers, are out of scope and switched off. Prerequisites: derivative and partial derivative of a polynomial; no programming is required.
Available Approach
Existing prototype.: We proceed with a visual 3D Gradient Descent Lab (<gradient.stergios.ch>), a Django application with Three.js graphics where all computation runs in the browser. Relevant features:
* split-screen layout: a narrow control panel (missions, surface selector, learning-rate slider, step controls) and a large 3D canvas with a ball rolling along the gradient descent trajectory, with a toggle to a 2D contour map showing gradient directions;
* five gamified missions of increasing difficulty (convex bowl, saddle, Rosenbrock ravine, multi-modal Himmelblau, Rastrigin), plus procedurally generated random surfaces;
* hover cards that explain each parameter (equation, intuition, ``too low'' vs.\ ``too high'' consequences);
* automatic telemetry: pseudonymous session ID plus logged inputs, hyperparameters, and outcomes (stored in SQLite).
Learning mechanisms (course toolbox).:
\begin{center} \small \begin{tabular}{@{}>{\raggedright\arraybackslash}p{3.0cm} >{\raggedright\arraybackslash}p{12.7cm}@{}} \toprule Mechanism & How it appears in the activity\ \midrule Contrasting cases & Runs with small, medium, and large learning rates on the same surface; bowl vs.\ ravine.\ Undoing & Predict-then-observe tasks expose the ``bigger is faster'' misconception.\ Self-explanation & After a run, learners write why the trajectory behaved as it did.\ Feedback & Information on the correctness of predictions and how to improve.\ Deliberate practice & Repeated missions with varying surfaces and starting points.\ \bottomrule \end{tabular} \end{center}
Study structure.: Following our earlier task analysis, learners first work through practice missions with the support of their condition and then solve test missions without any support in both groups, so that we measure learning rather than performance with help.
Condition A: no AI (control).: The prototype as built: missions, score, and the fixed hover explanation cards as generic rules of how parameters affect performance. Prediction and self-explanation prompts are present, but answers receive only fixed, pre-written feedback.
Condition B: AI-scaffolded.: Same environment and missions, plus an AI assistant in a side panel that analyzes the learner's predictions and explanations and interacts through dialogue. Design rules, taken from our team agreement:
* no direct answers and no computation of the update for the learner; only guided hints, Socratic questions, and high-level suggestions;
* prompts that ask learners to articulate reasoning and notice gaps (*problematizing* scaffolding, not only *structuring*);
* access to the current simulation state (learning rate, trajectory, loss), fixed system prompt, and full logging.
This keeps the AI in service of the learner's productive effort rather than replacing it.
Still to build (next steps).: (i) AI side panel and system prompt; (ii) predict-first and self-explanation prompts in both conditions; (iii) test missions on unseen surfaces without support; (iv) integration of pre-/post-test items and surveys; (v) a switch that disables optimizer variants and unused levels. Stergios leads the implementation; Isna, together with the TA, defines the pre-screening and pre-test instrument.
Open design decision.: As an alternative or later extension we could compare AI roles, AI as tutor (expert guidance) vs.\ AI as peer (co-learner). With about 20 participants we keep two groups (10 vs.\ 10) and avoid factorial designs, which would leave only five per cell.
Research Questions
\begin{description}[leftmargin=1.2cm, labelwidth=1cm, style=nextline] * [RQ1] Does adding an AI-based Socratic scaffold to an interactive gradient descent lab lead to higher post-test scores than the same lab with fixed feedback? * [RQ2] Does the AI-scaffolded condition lead to better performance on unassisted test missions and transfer items (surfaces not seen during practice)? * [RQ3 (exploratory)] How do learners use the assistant (e.g., asking for answers vs.\ explaining their reasoning), and how do usage, perceived cognitive load, and motivation relate to learning gain? \end{description}
We expect, but do not assume, that the AI condition helps mainly through more and better self-explanations. With around 20 participants the study is exploratory, and we report effect sizes and uncertainty rather than relying on significance alone.
Methodology
\label{sec:method}
Design.: Between-subjects pre-test / intervention / post-test study with two conditions (A: no AI, B: AI-scaffolded). A pre-quiz is administered before grouping, and participants are randomly assigned, stratified by pre-test score so that prior knowledge is balanced.
Participants and recruitment.: About 20 EPFL Bachelor students (10 per condition), recruited through contacts of the team (second-year Bachelor students) and course channels. A short pre-screening test filters genuine novices. A pilot with the students of an upcoming lab session (about 50) can test the tool and the items, but pilot data are not used as study data.
Procedure: (about 60 minutes, individual sessions):
* Consent, background questionnaire (prior knowledge, AI literacy) and pre-test --- 10 min
* Practice missions in the assigned condition, same tasks and time limit --- 25 min
* Unassisted test missions on unseen surfaces (no hints, no AI, no hover cards) --- 10 min
* Post-test (parallel to the pre-test, plus transfer items) and survey --- 15 min
Pre- and post-test items are parallel versions (different numbers and functions, same structure) and counterbalanced across participants.
Measures.:
* *Learning (primary):* post-test score and normalized gain, scored with a rubric per objective (justifications scored 0--2 by two raters).
* *Unassisted test missions:* success, number of steps to reach the target, and time, relative to the difficulty of the surface (complexity vs.\ performance).
* *Process (from telemetry):* runs per mission, share of predictions made before running, time between interactions as an engagement indicator, and, in condition B, number and type of AI messages.
* *Survey:* AI literacy, intrinsic motivation, perceived cognitive load, and perceived helpfulness, kept separate from learning since perceived and actual learning can diverge.
* *Optional delayed measure:* performance on the related assignment in the machine learning course, if the course staff agree to share it.
Analysis.: Post-test and test-mission scores are compared between conditions with ANCOVA using the pre-test as covariate, plus a non-parametric check (Mann--Whitney U); we report effect sizes with confidence intervals. AI dialogues and self-explanations are coded qualitatively (request for answer, reasoning attempt, misconception expressed) by two coders with agreement reported. Process measures are interpreted only in relation to the learning outcome.
Threats and mitigations.:
* *Small sample:* analysis plan and items are fixed in advance; results are exploratory.
* *Gamification:* the score can reward trial-and-error tuning instead of reasoning. Both groups see the same score, predict-first prompts are required, and the unassisted test missions check whether reasoning transferred.
* *AI giving away answers:* the system prompt is tested with adversarial inputs and all outputs are logged.
* *Time on task and rater bias:* fixed time limits; raters are blind to condition.
Ethics and data.: The current telemetry records every visitor automatically. For the study we will obtain informed consent that explicitly describes this logging, keep only pseudonymous IDs, and store data on a protected server. The AI interaction logs of the team's own work are submitted with the milestones as required by the course.
Timeline.:
* Oct 8 (M3): project plan and first working prototype; TA validation of idea and recruitment pipeline
* Autumn break to week 8: implement AI panel and test missions, pilot, finalize items and pre-screening test
* Nov 5 (M4): study plan and finished implementations for both conditions
* Weeks 9--12: run about 20 sessions (about 30 hours of testing including set-up)
* Weeks 13--14: analysis; Dec 15: presentation; Jan 7: final report
* Mid-January 2027: paper for IJCAI 2027 (Human-Centred AI track; abstract about Jan 12, full paper about Jan 19), emphasizing the human--AI interaction design, error correction, and cognitive load / transfer