Unplugged Teaching: DESI 2025 vs 2026 (A/B Empirical Evaluation)

Unplugged Teaching: DESI 2025 vs 2026 (A/B Empirical Evaluation)

Project Leads: Stergios Konstantinidis & James
Affiliation: Department of Information Systems (DESI), Faculty of Business and Economics (HEC Lausanne), University of Lausanne (UNIL)
Initiative: DESI Cross-Team Research Collaboration
Primary Source Document: unplugged_teaching___desi_2025_vs_2026-7.pdf (Local path: /Users/stergios/Downloads/unplugged_teaching___desi_2025_vs_2026-7.pdf)
Target Venues: ACM Transactions on Computing Education (TOCE) / IEEE Transactions on Learning Technologies (TLT) / ECIS Education Track / ICIS

---

1. Executive Summary & Context

Advances in artificial intelligence (AI) have fundamentally reshaped university education and learning environments. Within the Department of Information Systems (DESI) at HEC Lausanne / UNIL, this technological transition reached a pivotal turning point when students raised concerns to teaching staff regarding outdated pedagogical methodologies in an era characterized by ubiquitous AI assistance.

In direct response to these concerns, the department made the strategic institutional decision to introduce unplugged lecture sessions (structured in-class periods restricting laptop/device and LLM usage to focus on conceptual grounding, interactive discussions, and active problem-solving).

This research project seizes this institutional policy shift to conduct a rigorous, real-world quasi-experimental A/B evaluation, systematically comparing the 2025 cohort (pre-unplugged baseline) against the 2026 cohort (unplugged lecture intervention).

The project is a direct continuation and expansion of the DESI PhD Cross-Team Publication Initiative, fostering cross-disciplinary synergy across departmental research groups.

flowchart TD
    subgraph S1["Institutional Catalysts"]
        A1["Ubiquitous Generative AI Adoption"] --> A2["Student Concerns on Legacy Teaching Methods"]
        A2 --> A3["DESI Department Decision: Unplugged Lecture Sessions"]
    end

    subgraph S2["Quasi-Experimental Design"]
        A3 --> B1["Cohort 2025 (Control Baseline)"]
        A3 --> B2["Cohort 2026 (Intervention / Unplugged)"]
    end

    subgraph S3["Multi-Dimensional Evaluation"]
        B1 & B2 --> C1["(a) Student Satisfaction & Well-being"]
        B1 & B2 --> C2["(b) Academic Performance & AI Dependency"]
        B1 & B2 --> C3["(c) In-Class Attention & Active Engagement"]
    end

    subgraph S4["Deliverables"]
        C1 & C2 & C3 --> D1["DESI Institutional Policy Guidelines"]
        C1 & C2 & C3 --> D2["Peer-Reviewed Archival Journal Publication"]
    end

---

2. Core Research Questions & Evaluation Dimensions

The study investigates three complementary, interdependent dimensions:

1. Overall Student Satisfaction:

   • Institutional course evaluations and standardized student feedback questionnaires.
   • Perceived pedagogical value of unplugged vs. digitally saturated lecture sessions.
   • Student sentiment regarding cognitive load, screen fatigue, and learning environment satisfaction.

2. Student Academic Performance & AI Dependency:

   • Objective academic performance: Average course grades, exam score distributions, and assignment quality.
   • Self-reported and measured reliance on Generative AI (LLMs) for coursework completion.
   • Professor evaluations of student mastery, critical thinking, and independent problem-solving depth.

3. In-Class Engagement & Classroom Dynamics:

   • Active participation, frequency of spontaneous question-asking, and collaborative peer interactions.
   • Distraction levels and presence of off-task digital multi-tasking behaviors.
   • Independent faculty assessments of student attention, responsiveness, and classroom vibrancy.

---

3. Quasi-Experimental A/B Methodology

Aspect	Cohort 2025 (Baseline / Control)	Cohort 2026 (Intervention / Unplugged)
Pedagogical Format	Standard lectures with unrestricted digital device and AI access	Structured mandatory "unplugged" lecture periods
Sample Population	Full DESI enrolled student cohort (2025)	Full DESI enrolled student cohort (2026)
Primary Data Sources	Course surveys, grade distributions, faculty surveys	Course surveys, grade distributions, faculty surveys
Longitudinal Follow-up	Recent graduate surveys at +6 mo and +12 mo post-degree	Recent graduate surveys at +6 mo and +12 mo post-degree
Faculty Feedback	Confidential retrospective & session engagement surveys	Confidential prospective session engagement surveys

---

4. Multi-Wave Longitudinal Timeline (2026 – 2028)

The project concludes when the 2026 cohort graduates and integrates into the workforce, spanning six distinct phases:

Milestone Date	Target Cohort	Phase & Objective	Key Deliverables & Survey Instruments
October 2026	Cross-Cohort	Ethics Application	Submission to UNIL Institutional Review Board / Ethics Committee
November 2026	2025 Cohort	Baseline Data Collection	Institutional course evaluations, course grade averages, faculty engagement surveys
Dec 2026 – Jan 2027	2025 Cohort	Wave 1 Graduate Survey	Post-graduation survey to winter 2025 graduates (preparedness, reflection)
June – July 2027	2025 Cohort	Wave 2 Graduate Survey	Follow-up survey to summer 2025 graduates (retention, professional application)
August 2027	2026 Cohort	Intervention Data Collection	Institutional course evaluations, course grade averages, faculty engagement surveys
Dec 2027 – Jan 2028	2026 Cohort	Wave 1 Graduate Survey	Post-graduation survey to winter 2026 graduates (unplugged cohort perception)
June – July 2028	2026 Cohort	Wave 2 Graduate Survey	Follow-up survey to summer 2026 graduates (longitudinal retention & concept mastery)
Autumn 2028	Joint	Synthesis & Paper Submission	Econometric modeling, cross-cohort statistical analysis, journal manuscript submission

---

5. Research Governance & Ethical Safeguards

1. IRB & Ethics Approval:
   • Formal application scheduled for October 2026 prior to any faculty or graduate data collection.
   • Explicit participant consent protocols for all student and faculty surveys.
2. Double-Blind Anonymization:
   • Student academic records and course grades will be aggregated and de-identified. No individual student identifying information will be stored or published.
3. Confidential Faculty Surveys:
   • Teaching staff and professors will be contacted independently and surveyed confidentially to preserve candid assessment and full institutional objectivity.
4. FAIR Data & Open Science:
   • De-identified survey instruments, psychometric measurement scales, and statistical analysis scripts will be deposited openly upon paper publication.

---

6. Project Team & How to Take Part

• Project Co-Leads: Stergios Konstantinidis & James.
• Participation Opportunities:
  • Co-Researchers & Analysts: DESI PhD students, postdocs, and faculty interested in survey design, psychometrics, statistical analysis, or co-authoring the publication.
  • Teaching Staff Contributors: Course coordinators participating via confidential engagement surveys.
• Onboarding Process:
  • Feel free to reach out to Stergios if you are interested in participating as a researcher.
  • A coffee or lunch will be organized to establish expectations, discuss specific sub-hypotheses, and outline the project plan.

---

7. Attached Materials & Primary Documents

• 📄 Executive Summary PDF: unplugged_teaching___desi_2025_vs_2026-7.pdf (Attachment ID: 19)
• 📥 Download: The full 1-page executive summary PDF is attached to this BookStack page and downloadable directly from the right-hand sidebar under Attachments or via the direct URL above.