105 lines
4.1 KiB
Markdown
105 lines
4.1 KiB
Markdown
# OCR Processing Specification
|
||
|
||
## Purpose
|
||
|
||
Define safe OCR behavior.
|
||
|
||
## Requirements
|
||
|
||
### Requirement: Per-Page Native Detection
|
||
|
||
The system MUST evaluate PDF pages independently. Native text is sufficient only when `N >= 120`, `A >= 80`, `W >= 20`, and `R <= 0.01`, under `pdf-detection-v1`.
|
||
|
||
#### Scenario: Textual pages bypass OCR
|
||
|
||
- GIVEN every page meets native thresholds
|
||
- WHEN extraction completes
|
||
- THEN native text MUST be selected and no OCR job MAY be created
|
||
|
||
#### Scenario: Mixed PDF selects exact pages
|
||
|
||
- GIVEN only some pages fail any native threshold
|
||
- WHEN detection completes
|
||
- THEN OCR MUST receive those unique, ordered, one-based pages
|
||
- AND `N == 0` MUST always request OCR inspection
|
||
|
||
### Requirement: Blank and OCR Quality Gates
|
||
|
||
A page MAY be blank only with `inkCoverage < 0.015` and fewer than 10 OCR characters. Non-blank OCR MUST meet `nonWhitespaceCharacters >= 40`, median `>= 0.80`, p10 `>= 0.50`, and low-confidence ratio `<= 0.20`.
|
||
|
||
#### Scenario: Verified blank contributes no text
|
||
|
||
- GIVEN both blank conditions hold
|
||
- WHEN the page is classified
|
||
- THEN it MUST be blank and contribute no candidate text
|
||
|
||
#### Scenario: Visible ink fails closed
|
||
|
||
- GIVEN visible ink and OCR fails the quality gate
|
||
- WHEN processing completes
|
||
- THEN the page MUST NOT be blank and the version MUST fail with `OCR_QUALITY_BLOCKED`
|
||
|
||
### Requirement: Deterministic Candidate Composition
|
||
|
||
The system MUST use native text or OCR lines, never both per page. It SHALL order lines by top, left, and original index; pages ascending; and separate non-blank pages with `--- Page N ---`.
|
||
|
||
#### Scenario: Candidate is reproducible
|
||
|
||
- GIVEN identical validated page records
|
||
- WHEN the candidate is composed repeatedly
|
||
- THEN canonical text, ordering, and hashes MUST be identical
|
||
- AND raw OCR and native extraction MUST remain available for audit
|
||
|
||
### Requirement: Risk Tokens Remain Uncorrected
|
||
|
||
The system MUST mark `\b[A-Za-z]{2,}[A-Za-z0-9_-]*\d[A-Za-z0-9_-]*\b` tokens and elevate ambiguous, differing, unique, or context-adjacent ones. It MUST NOT auto-correct casing or `O/0`, `I/1/l`, or `S/5`.
|
||
|
||
#### Scenario: Known code ambiguities are flagged
|
||
|
||
- GIVEN OCR emits `CBGO4a`, `FATo7`, `DSAuo8`, or `NSAvo6`
|
||
- WHEN risk analysis runs
|
||
- THEN each token MUST be shown for human review unchanged
|
||
|
||
### Requirement: Private Idempotent OCR Service
|
||
|
||
Private PaddleOCR `3.4.0` with PaddlePaddle CPU `3.2.2` MUST require bearer authorization, allowlisted jobs, and deterministic identity for identical hash, config, and pages. It MUST NOT be Internet-accessible.
|
||
|
||
#### Scenario: Idempotent submission
|
||
|
||
- GIVEN valid requests share a key and payload
|
||
- WHEN both are submitted
|
||
- THEN both MUST identify one job and result without duplicate processing
|
||
|
||
#### Scenario: Conflicting or invalid submission
|
||
|
||
- GIVEN a reused key conflicts, authentication fails, or configuration is disallowed
|
||
- WHEN submission occurs
|
||
- THEN OCR MUST return the contracted `409`, `401/403`, or `400` class
|
||
|
||
#### Scenario: Result integrity mismatch
|
||
|
||
- GIVEN returned integrity fields differ from the request
|
||
- WHEN RAG validates the result
|
||
- THEN processing MUST fail without a reviewable candidate
|
||
|
||
### Requirement: Operational Limits and Health
|
||
|
||
OCR MUST enforce 50 MiB, 100 pages, 200 DPI, 25 megapixels/page, one concurrent job, one replica, queue depth three, 60 seconds/page, 15 minutes total, and 3 CPU/5 GiB maximum. Readiness MUST await pinned models; polling SHALL back off 2–15 seconds.
|
||
|
||
#### Scenario: Unsafe input is rejected before render
|
||
|
||
- GIVEN a PDF is oversized, encrypted, corrupt, too long, or decompression-unsafe
|
||
- WHEN it is accepted for processing
|
||
- THEN it MUST be rejected before full render without changing the active version
|
||
|
||
#### Scenario: Queue or transient service pressure
|
||
|
||
- GIVEN the queue is full or OCR is transiently unavailable
|
||
- WHEN RAG submits work
|
||
- THEN `429` MUST remain retryable; connection/`502`/`503` failures MUST allow at most two same-key resubmissions
|
||
|
||
#### Scenario: Deterministic failure is not retried
|
||
|
||
- GIVEN OCR returns `400`, `401`, `403`, `413`, `422`, or deterministic render failure
|
||
- WHEN RAG handles the response
|
||
- THEN it MUST NOT retry or switch engines
|