DocEng Short Paper Cost-Aware Human-LLM Collaboration for Post-OCR Corrections in Swiss Historical Newspapers Overview & Executive Summary Paper: DocEng Short Paper - Cost-Aware Human-LLM Collaboration for Post-OCR Corrections Authors: Stergios Konstantinidis, Hayman Lotfy, Michalis Vlachos (University of Lausanne) Venue: ACM Symposium on Document Engineering (DocEng) - Short Paper GitHub: https://github.com/Stergios-Konstantinidis/Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections Overleaf: https://git.overleaf.com/6a2cb93de1bec86186ee8613 1. Research Motivation & Core Problem Historical document archives, such as digitized centuries-old Swiss newspapers, suffer from substantial OCR degradation caused by aged paper, bleed-through, complex multi-column typography, and antique fonts. While modern Large Language Models (LLMs) excel at OCR correction, naive all-to-LLM pipelines suffer from three severe pitfalls: Financial Cost: Processing millions of archive pages with commercial LLM APIs is economically non-viable. Degradation of Clean Text: LLMs frequently "hallucinate" or modernize archaic French spellings on segments that were already correctly transcribed by the OCR engine. Severe OCR Failures: On heavily damaged regions, LLMs fabricate plausible-sounding text instead of preserving historical fidelity. This paper formulates OCR post-correction as an optimal budget allocation and routing problem uniting human domain experts and LLMs. 2. Key Contributions Three-Tier Human-LLM Collaboration: Explicit routing of each segment to either *(1) No Correction*, *(2) Automated LLM Correction*, or *(3) Human Review*. Regression-Guided Prioritization: Learned Lasso/Ridge router predicting expected Character Error Rate (CER) reduction \hat{Δ}_i before sending tokens to an LLM. Post-LLM Safeguard Layer: Classification mechanism detecting when LLM correction introduced regression or hallucination, diverting compromised outputs to human reviewers. Superior Budget Efficiency: Achieves 14% relative CER reduction over the All-LLM baseline with <5% human review budget. Methodology & Architecture 1. System Architecture: Three-Tier Collaborative Routing Preserving digitized historical archives requires mitigating transcription degradation without incurring unsustainable computational inference costs or inducing catastrophic hallucination. Naive application of Large Language Models (LLMs) across entire archival collections results in prohibitive expenses, vocabulary modernization, and frequent regression of correctly transcribed rare historical entities. Our architecture introduces a three-tier routing paradigm that dynamically allocates each text segment to one of three optimal treatment paths based on learned degradation profiles and empirical confidence safeguards: Figure 1: Collaborative Human-LLM Post-OCR Correction Architecture with Dynamic Confidence Routing and Post-Correction Safeguard Loop. Class 0: Bypass Route (Direct to Archive) Segments identified as already clean or high-confidence bypass LLM correction completely. Saves 100% token cost and eliminates any risk of stylistic hallucination or diacritic corruption. Class 1: Automated LLM Correction Moderately degraded segments are dispatched to historical prompting pipelines (Gemini / GPT / Llama) with few-shot archival examples and strict orthographic grounding constraints. Class 2: Priority Human Verification (<5%) Severe segmentation failures, empty OCR blocks, or segments intercepted by the post-correction safeguard are routed to professional archivists under a strict <5% human time budget. 2. Mathematical Formulation & Quality Metrics Let D = {d1, …, dn} denote a digitized archival corpus. Each document segment di possesses an original raw OCR transcription diraw, a candidate machine correction dicorr, and a ground-truth transcription dihuman. We quantify correction quality via the Character Error Rate (CER) reduction benefit Δi: Δi = CER(diraw, dihuman) − CER(dicorr, dihuman) Positive Benefit (Δi > 0): The machine correction successfully lowered transcription noise and improved readability. Negative Regression (Δi < 0): The LLM degraded accuracy, hallucinated false cognates, or altered archaic syntax. The post-correction safeguard is designed to catch these regressions. 3. Feature Extraction & LASSO Degradation Modeling Prior to invoking expensive generative models, the router extracts 54 lightweight features capturing multi-faceted document degradation without requiring ground truth annotations: Feature Family Extracted Indicators Routing Rationale Confidence Metrics Mean token confidence, lowest quartile confidence, token variance Low confidence triggers LLM or human verification Lexical & Orthographic Out-of-vocabulary ratio against 18th/19th c. Swiss French lexicons, symbol density Distinguishes historical spelling from OCR garbage characters Perplexity & Language Model CamemBERT cross-entropy loss, token log-likelihoods Captures ungrammatical syntax disruptions caused by broken line breaks Typographic & Layout Bounding box aspect ratio, average word length, uppercase-to-lowercase transitions Identifies broken column cuts and hyphenation splits Figure 2: LASSO Feature Selection Coefficients identifying the strongest predictive drivers of post-OCR correction benefit (Δi). 4. Safeguard Routing Curves & Verification Budget A critical vulnerability in all-LLM correction pipelines is the silent corruption of historical names and rare dates. To eliminate this risk, our framework deploys a lightweight Post-Correction Safeguard classifier evaluated on the pair (diraw, dicorr): Computes normalized Levenshtein distance, word count divergence, and proper noun preservation. If the safeguard predicts that the candidate correction introduced regressions (CER(dicorr) > CER(diraw)), the text is intercepted prior to writing to the database and escalated directly to human review. Figure 3: Empirical Safeguard Routing Boundaries balancing Character Error Rate reduction against the strict human verification budget (<5%). Key Architectural Takeaways Cost Reduction: Bypassing 60%+ of clean segments cuts API inference expenditure by more than half. Guaranteed Quality: The dual-gate architecture (Router + Safeguard) prevents hallucinated modernizations from ever corrupting the archival repository. Human Efficiency: Prioritizing human review exclusively on high-uncertainty and safeguard-flagged segments maximizes archival accuracy per reviewer hour. Dataset & Benchmark Results Experimental Benchmark & Evaluation 1. Archival Corpus Evaluated across 609 text segments spanning 45 digitized pages from 9 Swiss French-language historical periodicals (Canton of Vaud, 1733–1945): Periodical Era Issues Pages Segments Characteristics La Revue 1875–1945 4 5 139 19th/20th century serif, multi-column Feuille d'Avis 1762–1841 4 6 118 Antique typography, long 's' ligatures Mercure Suisse 1733–1748 3 5 92 18th-century early modern French Gazette de Lausanne 1804–1920 5 8 145 High density, complex layout Others (Tribune, etc.) 1880–1930 5 21 115 Mixed commercial & political press 2. Key Benchmark Results Error Rate Comparison (Tesseract OCR Base): Raw OCR Baseline: CER = 6.30\%, WER = 14.20\% All-LLM Baseline (GPT-4o / Gemini 1.5 Flash): CER = 2.92\% (Cost: 100% token budget) Selective Pre-Routing (Ours): CER = 3.12\% (Bypasses 40% of all segments!) Collaborative Safeguard (<5% Human Budget): CER = 2.49\% (14% relative reduction over All-LLM) Comparison with Routing Baselines: Random Routing: Weakest; frequently sends clean text and degrades it. OCR Confidence (Thresholding): Underperforms because Tesseract confidence scores poorly correlate with semantic error in archaic fonts. ConfBERT: Good detection but computationally heavy. Our Regression Router + Safeguard: Consistently establishes the Pareto frontier across all budget limits. Experimental Evaluation & Visual Benchmark Plots Historical Newspaper Segmentation Sample Figure 1: Representative historical archival newspaper segment illustrating layout artifacts and OCR degradation. CER Reduction Across Correction Strategies Figure 2: Character Error Rate (CER) reduction across baseline OCR, All-LLM, ConfBERT, and Collaborative Routing. OCR Engine Improvement Comparison Figure 3: Relative error reduction across historical OCR engines (Tesseract, Abbyy, and modern HTR). Strategy Performance Heatmap Figure 4: Comprehensive performance heatmap comparing LLM prompt strategies across historical periodicals. Codebase & Reproduction Guide Codebase Structure & Execution 1. Directory Overview paper/: LaTeX source files, TikZ diagrams ( overview_tikz.tex), figures, and presentation slides. code/plotting/: Scripts generating progressive performance curves and slide assets ( generate_ppt_assets.py). code/evaluation/: Metric computation (CER, WER, edit distance, cost modeling). ppt assets/: Rendered publication assets (PDF/PNG curves for baseline, random, ConfBERT, ours, safeguard, oracle). DocEng presentation.pptx: Official presentation slide deck. 2. Key Commands # Clone the repository git clone https://github.com/Stergios-Konstantinidis/Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections.git cd Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections # Generate publication and presentation assets python3 code/plotting/generate_ppt_assets.py # Overleaf synchronization (pull before modify, push after) git pull origin main git push origin main Presentation slides 📊 DocEng 2026 Conference Presentation Deck Official 10-slide presentation delivered at ACM DocEng 2026 (28.08.2026). 📥 Download PDF 📥 Download PPTX Slide Deck & Presentation Breakdown Slide 1: Title & Authors Slide 01 / 10 Speaker Note: Cost-Aware Human-LLM Collaboration for Post-OCR Corrections in Swiss Historical Newspapers (DocEng 2026, 28.08.2026). Presented by Stergios Konstantinidis, Hayman Lotfy, Prof. Michalis Vlachos. Slide 2: Motivation & Archival Scale Slide 02 / 10 Speaker Note: Building RAG systems for historical archives across 300+ years of newspapers, millions of pages, and 12 TB of raw high-resolution scan imagery. Slide 3: Why Document Cleaning is Not Straightforward Slide 03 / 10 Speaker Note: Post-OCR correction is essential for downstream retrieval, but traditional tools neglect historical linguistic shifts and archaic orthography. Slide 4: Heterogeneous Segment Utility Slide 04 / 10 Speaker Note: Different segments benefit very differently: Segment D gains +2.0% CER improvement with zero overcorrections, while Segment B suffers -3.0% CER degradation due to 10 overcorrections! Slide 5: Optimal Correction Prioritization Slide 05 / 10 Speaker Note: Theoretical premise: What if we knew in advance which archival documents would benefit most from LLM correction? Slide 6: The Real-World Dilemma Slide 06 / 10 Speaker Note: In reality, ground truth is unavailable. Naively correcting all documents produces unpredictable degradation and wastes massive token budgets. Slide 7: Our Contribution - Intelligent Routing Architecture Slide 07 / 10 Speaker Note: A machine-learned Router evaluating expected CER benefit: If LLM is sufficient -> route to LLM; if degradation is too severe -> route to human review (<5%); otherwise leave the segment untouched. Slide 8: Experimental Evaluation & Benchmark Curves Slide 08 / 10 Speaker Note: Empirical results comparing baseline OCR, All-LLM, ConfBERT, and our Cost-Aware Routing framework across historical Swiss periodicals. Slide 9: Key Takeaways & Research Streams Slide 09 / 10 Speaker Note: Key contributions: 40% reduction in correction effort, negligible setup cost, and an effective human guardrail identifying segments requiring expert archivist attention. Slide 10: Conclusion & Contact Slide 10 / 10 Speaker Note: Wrap-up, open questions, and research collaboration contact information at UNIL-DESI.