# OCR Ingest Orchestration Specification

## Purpose

Define routing, lifecycle, recovery, and release behavior for OCR ingestion without changing native ingestion.

## Requirements

### Requirement: Native and OCR Routing

The system MUST preserve the synchronous Point 2 path when all PDF pages have sufficient native text. Before accepting OCR work, it MUST retain the original and manifest durably and create a non-active version with a null source content hash.

#### Scenario: Native PDF remains synchronous

- GIVEN a PDF whose pages satisfy native detection
- WHEN it is ingested
- THEN the system MUST return the existing `201` response and MUST NOT contact OCR
- AND lifecycle and retrieval behavior MUST remain unchanged

#### Scenario: OCR work is accepted asynchronously

- GIVEN a valid scanned or mixed PDF requiring OCR
- WHEN ingestion is accepted
- THEN it MUST return `202` with source/version identity, `state: indexing`, `phase: ocr_queued`, and `statusUrl`
- AND `reviewUrl` MUST be null and `activated` MUST be false

### Requirement: Version-Level Progress Gate

The system MUST report authenticated document/page progress using `native_extracting`, `ocr_queued`, `ocr_running`, `review_required`, `indexing`, `ready`, `active`, `failed`, or `rejected`. A version SHALL reach `review_required` only after every document is native-complete or OCR-complete.

#### Scenario: Multi-document candidate completes

- GIVEN a version with native and OCR documents
- WHEN all non-blank pages pass their applicable extraction gates
- THEN the version MUST become `review_required` with a review URL
- AND no document MAY be omitted from reported progress

#### Scenario: One document fails

- GIVEN any requested page fails or violates integrity or quality gates
- WHEN version progress is evaluated
- THEN the version MUST become `failed` with an actionable error
- AND it MUST produce no embeddings or partial activation

### Requirement: Idempotent Concurrent Dispatch

The system MUST maintain one job per version/document, prevent concurrent ownership, and reuse its remote idempotency key during recovery.

#### Scenario: Duplicate ingestion while pending

- GIVEN the same source and original have a non-terminal OCR version
- WHEN ingestion is retried
- THEN it MUST return that version and job without duplicates

#### Scenario: Lease recovery

- GIVEN one job has a live lease and another has an expired lease
- WHEN dispatch or reconciliation runs concurrently
- THEN the live job MUST remain untouched and the expired job MUST be redispatched once with its original key
- AND an existing remote job MUST be recovered rather than duplicated

### Requirement: Fail-Closed Availability

OCR candidates MUST remain invisible until approved and active. OCR, catalog, embedding, or vector failures MUST preserve the previous active version.

#### Scenario: OCR is unavailable

- GIVEN OCR cannot complete after allowed transient retries
- WHEN the job is processed
- THEN the candidate MUST fail closed without activation or engine substitution
- AND the prior active version MUST remain retrievable

#### Scenario: Catalog is unavailable during retrieval

- GIVEN active-version resolution is unavailable
- WHEN retrieval is requested
- THEN the system MUST return `503` and MUST NOT query without an active-version filter

### Requirement: Controlled Enablement and Rollback

OCR MUST be enabled only after its private service is ready, RAG is deployed disabled, and native behavior is verified. Disabling it MUST stop new OCR work while preserving native ingestion and candidate invisibility.

#### Scenario: Release gate passes

- GIVEN OCR is ready and `npm run check`, `npm run build`, `npm test`, and offline OCR tests pass
- WHEN OCR is enabled after native production verification
- THEN new eligible ingestions MAY use the OCR path

#### Scenario: Operational rollback

- GIVEN OCR processing or acceptance fails after deployment
- WHEN OCR is disabled or the RAG release is rolled back
- THEN native ingestion MUST continue and existing candidates MUST remain non-retrievable
- AND the previous active corpus MUST require no embedding recomputation