# DocEng Short Paper
Cost-Aware Human-LLM Collaboration for Post-OCR Corrections in Swiss Historical Newspapers
# Overview & Executive Summary
**Paper:** DocEng Short Paper - Cost-Aware Human-LLM Collaboration for Post-OCR Corrections
**Authors:** Stergios Konstantinidis, Hayman Lotfy, Michalis Vlachos (University of Lausanne)
**Venue:** ACM Symposium on Document Engineering (DocEng) - Short Paper
**GitHub:** https://github.com/Stergios-Konstantinidis/Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections
**Overleaf:** https://git.overleaf.com/6a2cb93de1bec86186ee8613
---
## 1. Research Motivation & Core Problem
Historical document archives, such as digitized centuries-old Swiss newspapers, suffer from substantial OCR degradation caused by aged paper, bleed-through, complex multi-column typography, and antique fonts.
While modern Large Language Models (LLMs) excel at OCR correction, naive **all-to-LLM pipelines** suffer from three severe pitfalls:
1. **Financial Cost:** Processing millions of archive pages with commercial LLM APIs is economically non-viable.
2. **Degradation of Clean Text:** LLMs frequently "hallucinate" or modernize archaic French spellings on segments that were already correctly transcribed by the OCR engine.
3. **Severe OCR Failures:** On heavily damaged regions, LLMs fabricate plausible-sounding text instead of preserving historical fidelity.
This paper formulates OCR post-correction as an **optimal budget allocation and routing problem** uniting human domain experts and LLMs.
---
## 2. Key Contributions
- **Three-Tier Human-LLM Collaboration:** Explicit routing of each segment to either \*(1) No Correction\*, \*(2) Automated LLM Correction\*, or \*(3) Human Review\*.
- **Regression-Guided Prioritization:** Learned Lasso/Ridge router predicting expected Character Error Rate (CER) reduction *\\hat{Δ}\_i* before sending tokens to an LLM.
- **Post-LLM Safeguard Layer:** Classification mechanism detecting when LLM correction introduced regression or hallucination, diverting compromised outputs to human reviewers.
- **Superior Budget Efficiency:** Achieves **14% relative CER reduction** over the All-LLM baseline with **<5% human review budget**.
# Methodology & Architecture
## 1. System Architecture: Three-Tier Collaborative Routing
Preserving digitized historical archives requires mitigating transcription degradation without incurring unsustainable computational inference costs or inducing catastrophic hallucination. Naive application of Large Language Models (LLMs) across entire archival collections results in prohibitive expenses, vocabulary modernization, and frequent regression of correctly transcribed rare historical entities.
Our architecture introduces a **three-tier routing paradigm** that dynamically allocates each text segment to one of three optimal treatment paths based on learned degradation profiles and empirical confidence safeguards:
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-methodology-overview.png "Click to view full-resolution architecture diagram") **Figure 1:** Collaborative Human-LLM Post-OCR Correction Architecture with Dynamic Confidence Routing and Post-Correction Safeguard Loop.
#### Class 0: Bypass Route (Direct to Archive)
Segments identified as already clean or high-confidence bypass LLM correction completely. Saves 100% token cost and eliminates any risk of stylistic hallucination or diacritic corruption.
#### Class 1: Automated LLM Correction
Moderately degraded segments are dispatched to historical prompting pipelines (Gemini / GPT / Llama) with few-shot archival examples and strict orthographic grounding constraints.
#### Class 2: Priority Human Verification (<5%)
Severe segmentation failures, empty OCR blocks, or segments intercepted by the post-correction safeguard are routed to professional archivists under a strict <5% human time budget.
**Δ*i* = CER(*d**i*raw, *d**i*human) − CER(*d**i*corr, *d**i*human)**
- **Positive Benefit (Δ*i* > 0):** The machine correction successfully lowered transcription noise and improved readability.
- **Negative Regression (Δ*i* < 0):** The LLM degraded accuracy, hallucinated false cognates, or altered archaic syntax. The post-correction safeguard is designed to catch these regressions.
---
## 3. Feature Extraction & LASSO Degradation Modeling
Prior to invoking expensive generative models, the router extracts **54 lightweight features** capturing multi-faceted document degradation without requiring ground truth annotations:
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-lasso-feature-impact.png "Click to view full-resolution LASSO feature plot") **Figure 2:** LASSO Feature Selection Coefficients identifying the strongest predictive drivers of post-OCR correction benefit (Δ*i*).
---
## 4. Safeguard Routing Curves & Verification Budget
A critical vulnerability in all-LLM correction pipelines is the *silent corruption* of historical names and rare dates. To eliminate this risk, our framework deploys a lightweight **Post-Correction Safeguard** classifier evaluated on the pair (*d**i*raw, *d**i*corr):
- Computes normalized Levenshtein distance, word count divergence, and proper noun preservation.
- If the safeguard predicts that the candidate correction introduced regressions (*CER*(*d**i*corr) > *CER*(*d**i*raw)), the text is intercepted prior to writing to the database and escalated directly to human review.
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-safeguard-routing.png "Click to view full-resolution safeguard routing plot") **Figure 3:** Empirical Safeguard Routing Boundaries balancing Character Error Rate reduction against the strict human verification budget (<5%).
#### Key Architectural Takeaways
- **Cost Reduction:** Bypassing 60%+ of clean segments cuts API inference expenditure by more than half.
- **Guaranteed Quality:** The dual-gate architecture (Router + Safeguard) prevents hallucinated modernizations from ever corrupting the archival repository.
- **Human Efficiency:** Prioritizing human review exclusively on high-uncertainty and safeguard-flagged segments maximizes archival accuracy per reviewer hour.
# Dataset & Benchmark Results
---
# Experimental Benchmark & Evaluation
## 1. Archival Corpus
Evaluated across **609 text segments** spanning **45 digitized pages** from 9 Swiss French-language historical periodicals (Canton of Vaud, 1733–1945):
| Periodical | Era | Issues | Pages | Segments | Characteristics |
| **La Revue** | 1875–1945 | 4 | 5 | 139 | 19th/20th century serif, multi-column |
| **Feuille d'Avis** | 1762–1841 | 4 | 6 | 118 | Antique typography, long 's' ligatures |
| **Mercure Suisse** | 1733–1748 | 3 | 5 | 92 | 18th-century early modern French |
| **Gazette de Lausanne** | 1804–1920 | 5 | 8 | 145 | High density, complex layout |
| **Others (Tribune, etc.)** | 1880–1930 | 5 | 21 | 115 | Mixed commercial & political press |
---
## 2. Key Benchmark Results
### Error Rate Comparison (Tesseract OCR Base):
- **Raw OCR Baseline:** *CER = 6.30\\%*, *WER = 14.20\\%*
- **All-LLM Baseline (GPT-4o / Gemini 1.5 Flash):** *CER = 2.92\\%* (Cost: 100% token budget)
- **Selective Pre-Routing (Ours):** *CER = 3.12\\%* (Bypasses **40%** of all segments!)
- **Collaborative Safeguard (<5% Human Budget):** ***CER = 2.49\\%*** (**14% relative reduction** over All-LLM)
### Comparison with Routing Baselines:
1. **Random Routing:** Weakest; frequently sends clean text and degrades it.
2. **OCR Confidence (Thresholding):** Underperforms because Tesseract confidence scores poorly correlate with semantic error in archaic fonts.
3. **ConfBERT:** Good detection but computationally heavy.
4. **Our Regression Router + Safeguard:** Consistently establishes the Pareto frontier across all budget limits.
---
## Experimental Evaluation & Visual Benchmark Plots
### Historical Newspaper Segmentation Sample
### CER Reduction Across Correction Strategies
### OCR Engine Improvement Comparison
### Strategy Performance Heatmap
# Codebase & Reproduction Guide
---
# Codebase Structure & Execution
## 1. Directory Overview
- `paper/`: LaTeX source files, TikZ diagrams (`overview_tikz.tex`), figures, and presentation slides.
- `code/plotting/`: Scripts generating progressive performance curves and slide assets (`generate_ppt_assets.py`).
- `code/evaluation/`: Metric computation (CER, WER, edit distance, cost modeling).
- `ppt assets/`: Rendered publication assets (PDF/PNG curves for baseline, random, ConfBERT, ours, safeguard, oracle).
- `DocEng presentation.pptx`: Official presentation slide deck.
## 2. Key Commands
```
# Clone the repository
git clone https://github.com/Stergios-Konstantinidis/Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections.git
cd Cost-Aware-Human-LLM-Collaboration-for-post-OCR-Corrections
# Generate publication and presentation assets
python3 code/plotting/generate_ppt_assets.py
# Overleaf synchronization (pull before modify, push after)
git pull origin main
git push origin main
```
# Presentation slides
### 📊 DocEng 2026 Conference Presentation Deck
Official 10-slide presentation delivered at ACM DocEng 2026 (28.08.2026).
[ 📥 Download PDF ](https://wiki.stergios.ch/attachments/22) [ 📥 Download PPTX ](https://wiki.stergios.ch/attachments/23)
### Slide 1: Title & Authors
Slide 01 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-01.png "Click to inspect slide 1 in high resolution")
**Speaker Note:** Cost-Aware Human-LLM Collaboration for Post-OCR Corrections in Swiss Historical Newspapers (DocEng 2026, 28.08.2026). Presented by Stergios Konstantinidis, Hayman Lotfy, Prof. Michalis Vlachos.
### Slide 2: Motivation & Archival Scale
Slide 02 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-02.png "Click to inspect slide 2 in high resolution")
**Speaker Note:** Building RAG systems for historical archives across 300+ years of newspapers, millions of pages, and 12 TB of raw high-resolution scan imagery.
### Slide 3: Why Document Cleaning is Not Straightforward
Slide 03 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-03.png "Click to inspect slide 3 in high resolution")
**Speaker Note:** Post-OCR correction is essential for downstream retrieval, but traditional tools neglect historical linguistic shifts and archaic orthography.
### Slide 4: Heterogeneous Segment Utility
Slide 04 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-04.png "Click to inspect slide 4 in high resolution")
**Speaker Note:** Different segments benefit very differently: Segment D gains +2.0% CER improvement with zero overcorrections, while Segment B suffers -3.0% CER degradation due to 10 overcorrections!
### Slide 5: Optimal Correction Prioritization
Slide 05 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-05.png "Click to inspect slide 5 in high resolution")
**Speaker Note:** Theoretical premise: What if we knew in advance which archival documents would benefit most from LLM correction?
### Slide 6: The Real-World Dilemma
Slide 06 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-06.png "Click to inspect slide 6 in high resolution")
**Speaker Note:** In reality, ground truth is unavailable. Naively correcting all documents produces unpredictable degradation and wastes massive token budgets.
### Slide 7: Our Contribution - Intelligent Routing Architecture
Slide 07 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-07.png "Click to inspect slide 7 in high resolution")
**Speaker Note:** A machine-learned Router evaluating expected CER benefit: If LLM is sufficient -> route to LLM; if degradation is too severe -> route to human review (<5%); otherwise leave the segment untouched.
### Slide 8: Experimental Evaluation & Benchmark Curves
Slide 08 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-08.png "Click to inspect slide 8 in high resolution")
**Speaker Note:** Empirical results comparing baseline OCR, All-LLM, ConfBERT, and our Cost-Aware Routing framework across historical Swiss periodicals.
### Slide 9: Key Takeaways & Research Streams
Slide 09 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-09.png "Click to inspect slide 9 in high resolution")
**Speaker Note:** Key contributions: 40% reduction in correction effort, negligible setup cost, and an effective human guardrail identifying segments requiring expert archivist attention.
### Slide 10: Conclusion & Contact
Slide 10 / 10
[  ](https://wiki.stergios.ch/uploads/images/gallery/2026-09/doceng-slide-10.png "Click to inspect slide 10 in high resolution")
**Speaker Note:** Wrap-up, open questions, and research collaboration contact information at UNIL-DESI.