rag-service/openspec/specs/ocr-processing/spec.md

OCR Processing Specification

Purpose

Define safe OCR behavior.

Requirements

Requirement: Per-Page Native Detection

The system MUST evaluate PDF pages independently. Native text is sufficient only when N >= 120, A >= 80, W >= 20, and R <= 0.01, under pdf-detection-v1.

Scenario: Textual pages bypass OCR

• GIVEN every page meets native thresholds
• WHEN extraction completes
• THEN native text MUST be selected and no OCR job MAY be created

Scenario: Mixed PDF selects exact pages

• GIVEN only some pages fail any native threshold
• WHEN detection completes
• THEN OCR MUST receive those unique, ordered, one-based pages
• AND N == 0 MUST always request OCR inspection

Requirement: Blank and OCR Quality Gates

A page MAY be blank only with inkCoverage < 0.015 and fewer than 10 OCR characters. Non-blank OCR MUST meet nonWhitespaceCharacters >= 40, median >= 0.80, p10 >= 0.50, and low-confidence ratio <= 0.20.

Scenario: Verified blank contributes no text

• GIVEN both blank conditions hold
• WHEN the page is classified
• THEN it MUST be blank and contribute no candidate text

Scenario: Visible ink fails closed

• GIVEN visible ink and OCR fails the quality gate
• WHEN processing completes
• THEN the page MUST NOT be blank and the version MUST fail with OCR_QUALITY_BLOCKED

Requirement: Deterministic Candidate Composition

The system MUST use native text or OCR lines, never both per page. It SHALL order lines by top, left, and original index; pages ascending; and separate non-blank pages with --- Page N ---.

Scenario: Candidate is reproducible

• GIVEN identical validated page records
• WHEN the candidate is composed repeatedly
• THEN canonical text, ordering, and hashes MUST be identical
• AND raw OCR and native extraction MUST remain available for audit

Requirement: Risk Tokens Remain Uncorrected

The system MUST mark \b[A-Za-z]{2,}[A-Za-z0-9_-]*\d[A-Za-z0-9_-]*\b tokens and elevate ambiguous, differing, unique, or context-adjacent ones. It MUST NOT auto-correct casing or O/0, I/1/l, or S/5.

Scenario: Known code ambiguities are flagged

• GIVEN OCR emits CBGO4a, FATo7, DSAuo8, or NSAvo6
• WHEN risk analysis runs
• THEN each token MUST be shown for human review unchanged

Requirement: Private Idempotent OCR Service

Private PaddleOCR 3.4.0 with PaddlePaddle CPU 3.2.2 MUST require bearer authorization, allowlisted jobs, and deterministic identity for identical hash, config, and pages. It MUST NOT be Internet-accessible.

Scenario: Idempotent submission

• GIVEN valid requests share a key and payload
• WHEN both are submitted
• THEN both MUST identify one job and result without duplicate processing

Scenario: Conflicting or invalid submission

• GIVEN a reused key conflicts, authentication fails, or configuration is disallowed
• WHEN submission occurs
• THEN OCR MUST return the contracted 409, 401/403, or 400 class

Scenario: Result integrity mismatch

• GIVEN returned integrity fields differ from the request
• WHEN RAG validates the result
• THEN processing MUST fail without a reviewable candidate

Requirement: Operational Limits and Health

OCR MUST enforce 50 MiB, 100 pages, 200 DPI, 25 megapixels/page, one concurrent job, one replica, queue depth three, 60 seconds/page, 15 minutes total, and 3 CPU/5 GiB maximum. Readiness MUST await pinned models; polling SHALL back off 2–15 seconds.

Scenario: Unsafe input is rejected before render

• GIVEN a PDF is oversized, encrypted, corrupt, too long, or decompression-unsafe
• WHEN it is accepted for processing
• THEN it MUST be rejected before full render without changing the active version

Scenario: Queue or transient service pressure

• GIVEN the queue is full or OCR is transiently unavailable
• WHEN RAG submits work
• THEN 429 MUST remain retryable; connection/502/503 failures MUST allow at most two same-key resubmissions

Scenario: Deterministic failure is not retried

• GIVEN OCR returns 400, 401, 403, 413, 422, or deterministic render failure
• WHEN RAG handles the response
• THEN it MUST NOT retry or switch engines