# Apply Progress: OCR Ingest Integration

## Current State

- **Mode:** Strict TDD
- **Delivery:** Feature-branch-chain; Units 1–13 implemented as bounded slices; production acceptance remains pending
- **Completed tasks:** 1.1–1.4, 2.1–2.5, 3.1–3.4, 4.1–4.5, 5.1–5.4, 6.1–6.4, 7.1–7.3
- **Overall task progress:** 29/30 complete

## Unit 1: Migration

### Implementation Summary

- Added `migrations/002_ocr_review.sql` with durable OCR jobs, auditable page records, atomic review correction targets, and a partial unique index for non-terminal OCR identities.
- Added focused schema contract coverage in `tests/catalog/migration-002.test.ts`.
- Preserved `migrations/001_knowledge_lifecycle.sql` and all repository logic unchanged.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 1.2 | `tests/catalog/migration-002.test.ts` | Schema contract | N/A (new files) | Valid RED: 4/4 failed with `ENOENT` because migration 002 did not exist. An earlier syntax-invalid run was corrected and is not counted as RED. | 4/4 passed with migration 002 present. | 4 independent schema behaviors cover jobs, pages, corrections, and pending identity uniqueness. | None needed; SQL already matches the minimal authoritative contract. |

### Test Summary

- **Total tests written:** 4
- **Total tests passing:** 4
- **Layers used:** Schema contract (4)
- **Approval tests:** None — no existing production file was modified
- **Pure functions created:** 0

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `npx --no-install tsx --test tests/catalog/migration-002.test.ts` — exit 0; 4 tests, 4 passed, 0 failed. |
| Runtime harness command/scenario and exact result | N/A — the task forecast defines Unit 1 as schema-only, and the project has no PostgreSQL test harness or database-test dependency. Applying a live migration would require external database access outside this unit. |
| Rollback boundary | Delete `migrations/002_ocr_review.sql` and `tests/catalog/migration-002.test.ts`, revert task 1.2 to pending, and remove this Unit 1 progress/history entry. No existing migration or repository behavior must be reverted. |

### Additional Validation

- `npm run check` — exit 0.
- `git diff --check` — exit 0 after removing one trailing-space warning introduced in the history header.

### Deviations

None — the migration follows the proposal, specifications, design, and closed OCR persistence contract.

### Remaining Persistence Tasks

- [x] 1.1 Complete repository-level RED coverage for duplicate pending ingestion, lease recovery, and review transitions in Unit 2.
- [x] 1.2 Add migration 002 schema and pending OCR identity unique index.
- [x] 1.3 Add catalog repository candidates and leases in Unit 2.
- [x] 1.4 Refactor persistence code after Unit 2 reaches green.

### Review Boundary

- **Start:** Migration 001 already supplies lifecycle states and parent catalog tables.
- **End:** Migration 002 and its focused schema contract test are green.
- **Out of scope:** Repository methods, dispatch behavior, runtime PostgreSQL migration, Unit 2, commits, pushes, PRs, deployment, and secrets.
- **Rollback:** Remove only the two Unit 1 implementation files and their progress metadata.
- **Authored review budget:** 225 additions/deletions, below the 400-line budget for this autonomous slice.

## Unit 2: Repository

### Implementation Summary

- Added pending OCR identity lookup, atomic queue claiming, expired-lease recovery, and guarded review transitions to `CatalogRepository`.
- Added a focused fake-pool repository harness that covers duplicate reuse, empty queues, remote identity preservation, and invalid transitions.
- Corrected the canonical test script so it executes both root-level and nested test suites.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 1.1 | `tests/catalog/repository-ocr.test.ts` | Unit/fake pool | 4/4 existing tests passed | 0/6 with methods absent; then 5/6 while each completion gate was missing | 6/6 passed after task 1.3 | Found/not-found candidates, claimed/empty queue, expired lease, valid/invalid transition | Covered by 1.4 |
| 1.3 | `tests/catalog/repository-ocr.test.ts` | Unit/fake pool | 4/4 existing tests passed | Tests preceded production code | 6/6 passed | Multiple inputs and state paths exercised | Shared OCR row mapping extracted only after green |
| 1.4 | `tests/catalog/repository-ocr.test.ts` | Approval refactor | 6/6 before refactor | N/A — behavior-preserving approval baseline | 6/6 after refactor | Existing six cases preserved | Reusable aliased-column projection removed SQL duplication |

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/catalog/repository-ocr.test.ts` — exit 0; 6 tests passed, 0 failed. `NODE_ENV=test npx --no-install tsx --test tests/catalog/*.test.ts` — exit 0; 10 tests passed, 0 failed. |
| Runtime harness command/scenario and exact result | `NODE_ENV=test npx --no-install tsx --test tests/*.test.ts` — exit 0; 25 root tests passed, 0 failed. Canonical `npm test` — exit 0; 35 total tests passed, 0 failed, proving the 25 root and 10 nested tests execute together. |
| Rollback boundary | Revert Unit 2 changes in `src/modules/catalog/repository.ts`, delete `tests/catalog/repository-ocr.test.ts`, and revert the `package.json` test script; Unit 1 migration remains intact. |

### Validation and Boundary

- Corrective rerun for failed evidence revision `sha256:9433c0cd6628f36f4c96a27d41c6d429f4303d038ffe928f297367c274f21c2f`; fresh evidence revision `sha256:a19119f84381447f54a6223e2a203acaf9d3ae08b6ae8f585e3bd52cd510909b`.
- The failed canonical script expanded only nested tests after nested suites appeared. The minimal correction is `NODE_ENV=test tsx --test tests/*.test.ts tests/**/*.test.ts`.
- Focused repository suite passed 6/6; all catalog suites passed 10/10; explicit root suites passed 25/25; canonical `npm test` passed 35/35; `npm run check` and `git diff --check` exited 0.
- Native Unit 2 accounting was 511 changed lines. The maintainer approved `size:exception` exclusively for Unit 2; the corrective attempt has a separate 600-line remediation budget. All later units retain the normal 400-line budget.
- No design deviations. No Unit 3 work was started.

### Corrective TDD Evidence

| Correction | Safety Net / RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|
| Canonical test discovery | Failed evidence revision proved canonical `npm test` executed only 10 nested tests while the explicit root command executed 25 tests. | Updated only the package test argument list; canonical `npm test` passed 35/35. | Direct root 25/25 and nested catalog 10/10 results sum to and identify the canonical 35/35 run. | None needed — one package-script argument was added. |

## Unit 3: Extraction

### Implementation Summary

- Added a Node 22 fixture spike proving `pdf-parse` page callbacks backed by bundled PDF.js `2.0.550` preserve ordered pages, including a blank page.
- Added `parsePdfPages` with one-based page identity, raw native text, stable SHA-256 hashes, and fail-closed complete-page accounting.
- Preserved textual ingestion as data-only, excluded `CMakeLists.txt`, MDX, and shell lookalikes, and kept task 2.2 pending for Unit 4 detection/composition assertions.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 2.1 | `tests/parsers/pdf-pages.test.ts`; `scripts/spike-pdfjs.ts` | Runtime spike | 20/20 lifecycle service tests passed | Focused suite failed 0/1 because `parsePdfPages` did not exist; the spike and parser implementation were still absent. | Node 22.14.0 spike returned pages 1–3 exactly, including blank page 2. | Three pages prove non-empty, blank, and later-page ordering paths. | Replaced the over-budget direct dependency with the design-approved existing `pdf-parse` pagerender path; spike remained green. |
| 2.3 | `tests/parsers/pdf-pages.test.ts` | Unit/integration fixture | 20/20 lifecycle service tests passed | Focused suite failed 0/1 on the missing export before production code. | Final focused run passed 5/5 after implementation. | Covered page hashes/blank page, native PDF composition, non-PDF rejection, data-only text parsing, and unsupported lookalikes. | Focused suite remained 5/5 after consolidating on the existing parser dependency. |

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/parsers/pdf-pages.test.ts` — exit 0; 5 passed, 0 failed. |
| Runtime harness command/scenario and exact result | `NODE_ENV=test npx --no-install tsx scripts/spike-pdfjs.ts` — exit 0 on Node 22.14.0; PDF.js 2.0.550 returned three ordered pages with page 2 blank. |
| Rollback boundary | Revert `src/modules/parsers/parser-registry.ts`; delete `scripts/spike-pdfjs.ts`, `tests/parsers/pdf-pages.test.ts`, and `tests/fixtures/ocr/native-three-pages.pdf`; revert tasks 2.1/2.3 and this Unit 3 progress/history entry. Units 1–2 remain intact. |

### Validation and Boundary

- Canonical `npm test` passed 40/40; `npm run check` and `git diff --check` exited 0.
- Native authored slice before progress/history metadata: 225 additions/deletions; no package or lockfile delta remains.
- Start: Unit 2 persistence is complete. End: per-page native extraction and parser threat routing are green.
- Out of scope: detection thresholds, OCR selection, composition, risks, Unit 4 tasks 2.4/2.5, commits, pushes, PRs, deployment, and restricted paths.
- No specification deviation. The design-authorized `pdf-parse` pagerender fallback was selected only after its Node 22 fixture spike passed.

## Unit 4: Detection and Composition

### Implementation Summary

- Added pure per-page native detection under `pdf-detection-v1`, exact OCR page selection, verified-blank classification, and fail-closed OCR quality gates.
- Added deterministic candidate composition using exactly one source per page, ordered OCR lines, blank omission, canonical hashes, and unchanged risk tokens prioritized for review.
- Completed the remaining task 2.2 scenarios by combining the Unit 3 data-only/parser threat coverage with Unit 4 mixed-PDF, detection, blank, hash, and risk RED assertions.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 2.2 | `tests/ocr/detection.test.ts`; `tests/parsers/pdf-pages.test.ts` | Unit/integration fixture | Unit 3 parser suite passed 5/5 before edits. | Focused Unit 4 suite failed before production with `ERR_MODULE_NOT_FOUND` for `src/modules/ocr/detection.js`. | Focused suite passed 5/5 after detection and composition were added. | Real generated mixed PDF selected pages 2 and 3; `.txt`/`.md` stayed data-only and CMake/MDX/shell lookalikes stayed unsupported. | None needed; routing remains separated between parser support and the pure PDF OCR planner. |
| 2.4 | `tests/ocr/detection.test.ts` | Unit | N/A (new production file) | Threshold, mixed-page, blank, and quality assertions preceded production code. | Exact native, blank, and OCR quality boundaries passed in the 5/5 focused suite. | Every native and OCR threshold has a passing boundary plus a failing neighbor; visible ink with empty OCR is blocked. | Pure functions and named policy version were present at green; final focused suite remained 5/5. |
| 2.5 | `tests/ocr/detection.test.ts` | Unit | N/A (new production file) | Ordered composition, blank omission, canonical hash, audit-source, and risk assertions preceded production code. | Deterministic composition and risk prioritization passed in the 5/5 focused suite. | Reordered pages, tied OCR boxes, blank middle page, repeated token, and four known ambiguous tokens exercise distinct paths. | Reused shared canonical JSON and SHA-256 utilities; final focused suite and type check remained green. |

### Test Summary

- **Tests:** 5 written and focused-passing; 45 canonical (unit 4, integration fixture 1)
- **Approval tests:** None — new production files; **pure functions created:** 6

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/ocr/detection.test.ts` — exit 0; 5 tests passed, 0 failed. |
| Runtime harness command/scenario and exact result | `NODE_ENV=test npx --no-install tsx --test --test-name-pattern "parsed mixed PDF" tests/ocr/detection.test.ts` — exit 0; 1 test passed, 0 failed. The harness generated a three-page PDF, parsed it through the Node 22 `pdf-parse` page callback, and selected only blank page 2 and insufficient page 3 for OCR. |
| Rollback boundary | Delete `src/modules/ocr/detection.ts`, `src/modules/ocr/composition.ts`, and `tests/ocr/detection.test.ts`; revert tasks 2.2/2.4/2.5 and this Unit 4 progress/history entry. Units 1–3 and their parser threat coverage remain intact. |

### Validation and Boundary

- Canonical `npm test` passed 45/45; `npm run check`, tracked `git diff --check`, and no-index whitespace checks for all three new files exited 0.
- Authored Unit 4 code and tests total 322 additions and 0 deletions, below the 400-line budget with no size exception.
- Start: Unit 3 page extraction and parser threat routing are complete. End: detection, blank/quality gates, deterministic composition, hashes, and risk tokens are green.
- Out of scope: OCR service, client, dispatcher, ingest routing, review APIs, commits, pushes, PRs, deployment, and restricted paths.
- No specification or design deviation.

## Unit 5: OCR Service

### Implementation and Correction Summary

- Added pinned Unit 5 FastAPI test dependencies and retained the ignored `ocr-service/.venv` environment.
- Added RED-first HTTP tests and a SQLite-backed private API for bearer auth, idempotency, allowlisting, upload/page limits, queue pressure, status, deletion, and health.
- Preserved failed evidence `sha256:4a5c8ed8d8b302f7ff654d4f40fe0f13049bfdd29db4c76576f3847c0a9be15a`: its harness returned `401/400/400`; inspection showed `&` backgrounded the preceding shell AND-list, so request variables existed only in the child shell and authenticated curls sent an empty `request` form, reproduced as `400 INVALID_REQUEST`.
- Corrected only the harness command boundary; the 381-line candidate implementation remained unchanged before progress persistence.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 3.1 | `ocr-service/tests/test_api.py` | HTTP integration | N/A (new service) | Collection failed with `ModuleNotFoundError: No module named 'app'`; failed localhost runtime evidence then supplied correction RED. | 8/8 focused tests and corrected curl `401/202/409` passed. | Auth variants, idempotency/conflict, allowlist, limits, pressure, integrity, and real multipart requests exercised distinct paths. | No production refactor: the defect was shell variable scope in the harness. |
| 3.2 | `ocr-service/tests/test_api.py` | HTTP integration/SQLite | N/A (new service) | Same missing-service RED covered the API; failed harness blocked completion. | 8/8 focused tests and real Uvicorn harness passed. | Ready/not-ready health, queue capacity, status, deletion, and process cleanup exercised distinct paths. | Pure page hashing and centralized terminal errors retained; no correction needed. |

### Work Unit Evidence

- Focused: `ocr-service/.venv/bin/python -m pytest ocr-service/tests -k auth` — exit 0; 8 passed, 0 failed (one dependency deprecation warning).
- Runtime: local Uvicorn plus multipart curl — unauthorized `401` with `UNAUTHORIZED`, accepted `202` with queued identity, conflicting same key `409` with `IDEMPOTENCY_CONFLICT`.
- Gates: `npm test` 45/45, `npm run check`, Python `compileall`, tracked and untracked whitespace checks — all exit 0/equivalent clean.
- Cleanup: Uvicorn PID 268425 terminated and absent; temporary PDF, SQLite DB, and log removed; `.venv` retained.
- Rollback: remove `.gitignore` Unit 5 entries and `ocr-service/`, then revert tasks 3.1/3.2 and this Unit 5 progress/history block; Units 1–4 remain intact.
- Fresh evidence revision: `sha256:0d8b2c57bbab967473eaf99a7ca900168554e64254fa80197b3626b4570b58d0`; corrected final authored change count: 397 additions plus deletions, within 400.

## Unit 6: OCR Rendering and Image

### Implementation Summary

- Added deterministic 200 DPI PDF page rendering with a pre-render 25-megapixel guard, a PaddleOCR adapter, stable line identities, quality metrics, and the exact result schema.
- Added a deterministic fake-driven test seam and preserved ambiguous OCR tokens without correction.
- Added the private CPU image with PaddleOCR `3.4.0`, PaddlePaddle CPU `3.2.2`, baked Latin/mobile models, one Uvicorn worker, a non-root user, private job storage, healthcheck, and documented 3 CPU/5 GiB deployment limits.
- Added a Dockerfile-specific deny-by-default build context so restricted and unrelated repository paths are never sent to the Docker daemon.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 3.3 | `ocr-service/tests/test_render.py` | Unit/integration fixture | Existing OCR API suite passed 8/8 before production edits. | Focused collection failed because `app.engine` did not exist. | Focused render suite passed 5/5; complete OCR suite passed 13/13. | Selected pages 1/3, invalid page and pixel-limit paths, repeatable fake results, and Paddle payload normalization exercise distinct behavior. | Named constants and pure metric/result transforms retained; focused suite remained 5/5. |
| 3.4 | `ocr-service/tests/test_render.py` | Static/container runtime | Existing OCR API suite passed 8/8 before production edits. | Docker contract was absent; deny-by-default context triangulation then failed until `Dockerfile.dockerignore` existed. | Image built with baked models; offline constrained container rendered and validated PNG/result output. | Static pin/runtime assertions plus real model build and `--network none` execution cover configuration and runtime paths. | Added missing OpenCV runtime libraries after the first build exposed `libGL.so.1`; exact corrected image passed. |

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `ocr-service/.venv/bin/python -m pytest ocr-service/tests -k render -q` — exit 0; 5 passed, 8 deselected, 1 dependency deprecation warning. |
| Runtime harness command/scenario and exact result | `docker build --file ocr-service/Dockerfile --tag rag-ocr-service:unit6 .` — corrected exact image exited 0 and baked both models. A `docker run --rm --network none --cpus 3 --memory 5g ...` harness loaded cached PaddleOCR 3.4.0/PaddlePaddle 3.2.2 models, rendered a 1700x2200 PNG (23,309 bytes), returned schema `1`, one page, and one OCR line; host Pillow reopened it as PNG 1700x2200. |
| Rollback boundary | Remove `ocr-service/{Dockerfile,Dockerfile.dockerignore,app/engine.py,app/models.py,app/render.py,tests/test_render.py}` and revert Unit 6 changes in `app/main.py`, `requirements.txt`, and `README.md`; Units 1–5 remain intact. |

### Validation and Boundary

- RED was followed by GREEN and post-refactor reruns. One expected GREEN iteration corrected a test-side non-whitespace count from 16 to the mathematically correct 15.
- The first image build failed on missing `libGL.so.1`; adding the required slim-image runtime libraries produced a successful build and runtime harness. A later optional connectivity-check refactor rebuild exhausted Docker storage; that unvalidated one-line refactor was reverted, the validated Dockerfile bytes were restored, and the image tag was removed during cleanup.
- `npm test` passed 45/45; `npm run check`, Python `compileall`, tracked/untracked whitespace checks, and the full OCR pytest suite (13/13) passed.
- Start: Unit 5 private queue/API is complete. End: Unit 6 render/engine/schema/container behavior is complete. Unit 7 client work was not started.
- Native Unit 6 implementation/tests/docs are 397 authored additions plus deletions, within the 400-line budget; required SDD progress and workspace history metadata are administrative evidence outside that native slice.
- No specification or design deviation. The Docker image is private by deployment contract; CPU/RAM labels document limits that the platform must enforce.

## Unit 7: OCR Client and Durable Artifacts

### Implementation Summary

- Added a private OCR HTTP client that preserves the exact idempotency key across one initial submission and at most two transient resubmissions, treats `429` as retryable queue pressure without immediate resubmission, and treats deterministic failures as terminal.
- Added 2–15 second polling backoff plus strict acknowledgement, status, engine, job, document, ordered-page, metrics, line, and schema validation before a result can be persisted.
- Added durable `0600` original and canonical manifest writes, UUIDv5 document artifact identities, lexical path containment, verifiable hashes, and age-gated orphan sweeping that preserves retained, recent, non-UUID, and symlink entries.
- Left task 4.1 pending because Unit 7 proves only its retries/integrity clause; Unit 8 still owns flag-off `201`, OCR `202`/status, OCR-down fail-closed, and catalog-down `503` HTTP coverage.

### TDD Cycle Evidence

| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 4.2 | `tests/ocr/client.test.ts` | Unit/runtime fetch stub | Existing OCR detection suite passed 5/5; production file was new. | Focused suite failed with `ERR_MODULE_NOT_FOUND` for `src/modules/ocr/client.js` before production code existed. | Final focused suite passed 6/6; runtime retry scenario passed 1/1. | Network then `503` produced exactly three same-key attempts with 2s/4s backoff; `429` and `422` each stopped after one attempt; malformed acknowledgement/schema/pages failed integrity; polling used 2s/4s. | Adapted multipart bytes to a type-safe `Uint8Array`; focused suite and type check remained green. |
| 4.3 | `tests/ocr/client.test.ts` | Filesystem integration | Existing OCR detection suite passed 5/5; production file was new. | The same missing-module RED preceded artifact production code. | Private originals, canonical manifest, safe resolution, and sweep passed in the final 6/6 suite. | Two out-of-order documents proved canonical ordering/hashes; traversal was rejected; sweep removed only an old orphan while preserving retained/recent directories and a symlink. | A test-side assertion was corrected to inspect the retained dangling symlink with `lstat`; no production behavior changed, and the suite passed 6/6. |

### Test Summary

- **Tests written and passing:** 6 focused tests; unit/fetch-stub and filesystem integration layers.
- **Approval tests:** None — both production modules are new.
- **RED:** exit 1, module-not-found before either production file existed.
- **GREEN/REFACTOR:** exit 0, 6 passed, 0 failed.

### Work Unit Evidence

| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/ocr/client.test.ts` — exit 0; 6 passed, 0 failed. |
| Runtime harness command/scenario and exact result | `NODE_ENV=test npx --no-install tsx --test --test-name-pattern "runtime retry stub" tests/ocr/client.test.ts` — exit 0; 1 passed, 0 failed. Deterministic stub observed network failure, `503`, then `202`: exactly 3 submission attempts, one unchanged idempotency key, 2,000/4,000 ms backoffs, followed by a strictly validated two-page result. |
| Rollback boundary | Remove `src/modules/ocr/client.ts`, `src/modules/ocr/artifacts.ts`, and `tests/ocr/client.test.ts`; revert tasks 4.2/4.3 and this Unit 7 progress/history metadata. Units 1–6 remain intact. |

### Validation and Boundary

- `npm test` passed 51/51; `npm run check`, `npm run build`, tracked whitespace, and no-index whitespace checks for all three new files passed.
- Test-created `rag-ocr-artifacts-*` and `rag-ocr-sweep-*` directories were removed; no server, external OCR call, or persistent process was started; `ocr-service/.venv` was preserved.
- Start: Unit 6 OCR rendering and container runtime are complete. End: Unit 7 client retries/integrity and durable artifacts/orphan sweep are implemented and green.
- Out of scope and untouched: dispatcher, reconciler, ingest branch, app upload/status routes, Unit 8 tests, commits, pushes, PRs, native review, deployment, and restricted paths.
- The implementation/test slice is 506 authored additions (client 174, artifacts 129, tests 203). This exceeds the 400-line default after one honest cohesive assessment; no code-golf or second slicing pass was attempted. The maintainer explicitly approved `size:exception`, including 585 total native-accounting lines with required metadata.
- No specification or design deviation.

### Approved Size-Exception Corrective Context

- The native attempt was reset after failed evidence revision `sha256:dd224c3864adc8380dc7d9fd147cb448f5da8138277f3f346a35d7eeb3aca158`; the failure was budget-only, not a functional defect.
- Corrective work unit `unit-7-size-exception-validation` uses attempt token `sha256:0109cfd8d3b37a5672c65a37700269b89cb80e4e357a39f68a920779a1715c39` and must settle against the failed revision through the parent.
- Production and test files remain unchanged for the corrective rerun; only approval/reset evidence and fresh validation metadata may change.
- Task 4.1 remains pending, while tasks 4.2 and 4.3 remain complete. Unit 8 was not started.

## Unit 8: Dispatcher and Runtime Orchestration
- Completed multi-document progress, uniqueness-race reuse, exact lease recovery, persisted-key dispatch, default runtime wiring, authenticated status, and fail-closed retrieval; no activation or embeddings occur on OCR failure.

### Evidence Chain and Review Accounting
- **Complete review slice:** Relative to committed HEAD `ac046e3`, Unit 8 contains 1,045 functional changed lines plus 41 required metadata lines, for 1,086 total changed lines. The complete slice is not within 400 lines, and no `size:exception` was approved or recorded for it.
- **Interrupted preserved candidate:** Native failed/interrupted revision `sha256:182cc73954a8518d03bd6c3ef5f4141b352aa9b69fca4177447d76deaddad223` contained 815 changed lines.
- **Auto-chain reset:** The maintainer-selected preflight strategy was `auto-chain` with `feature-branch-chain`. Native reset revision `sha256:c0db82cf3db8ae790d34e4c310b998f97bb36864c97c3aa84e4c87ab77d07c2a` preserved the interrupted candidate and scoped the next bounded recovery objective only to additional changes; it did not approve the complete Unit 8 slice as a size exception.
- **Recovery delta:** After that reset, the bounded recovery added 350 functional changed lines plus 41 required metadata lines, for 391 total changed lines within the separate 400-line recovery objective.
- **Passed recovery:** Fresh passed evidence revision `sha256:e424afe7b2f8efd547dcfd0baf64ec1b46ac344f39afd87259560a094d59163f` closes the interrupted → auto-chain reset → passed recovery chain.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 4.1 | `tests/ocr/{dispatcher,client}.test.ts` | HTTP/integration | Recovered baseline 7/7 passed; not labeled RED | New scenarios produced genuine 5/9 RED; catalog `/retrieve` already failed closed | Unit 8 9/9; client 6/6 | 201/202/status, OCR failure, 503 retrieval, retries/integrity | Consolidated persisted-key assertion; 9/9 |
| 4.4 | `tests/ocr/dispatcher.test.ts` | Unit/integration | Recovered baseline 7/7 | Exact recovery failed by claiming unrelated work | 9/9 | Live/new queue separation plus periodic dispatch | Exact-job dispatch helper; 9/9 |
| 4.5 | `tests/ocr/dispatcher.test.ts` | HTTP/filesystem | Recovered baseline 7/7 | Runtime wiring RED was 8/10 | 9/9 | Multipart upload, durable reload, duplicate race, mixed documents | Shared queue drain; 9/9 |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test | `NODE_ENV=test npx --no-install tsx --test tests/ocr/dispatcher.test.ts` — 9 passed, 0 failed. |
| Focused repository OCR test | `NODE_ENV=test npx --no-install tsx --test tests/catalog/repository-ocr.test.ts` — 6 passed, 0 failed. |
| Canonical Node test | `npm test` — 60 passed, 0 failed. |
| Runtime harness | Bounded localhost curl — ingest `202/ocr_queued`; authenticated status `200/ocr_queued`; leaked child from initial shell cleanup was detected, terminated, and port/temp cleanup reverified. |
| Rollback boundary | Revert Unit 8 deltas in `src/{app,config/env}.ts`, catalog/ingest/client modules, `src/modules/ocr/dispatcher.ts`, and its test; preserve Units 1–7. |

## Unit 9: Review and Indexing Core
- Added authenticated review/approval routes with injectable service boundaries, immutable correction preparation, stale/duplicate conflict guards, and new/reusable activation CAS behavior. Rejection transition and playground UI remain Unit 10.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 5.1 | `tests/ocr/review.test.ts` | HTTP/unit | App suites 9/9 and 20/20 | Missing `indexing.js`; exit 1 | 5/5 | Unauthorized/premature, candidate/line/active staleness, duplicate targets, new/reusable races, rejected no-index | Assertions remained 5/5 |
| 5.2 | `tests/ocr/review.test.ts` | HTTP/unit | 29/29 | Tests preceded `review.ts` | 5/5 | Audit view plus two corrections and four conflict paths | Clone prevents failed commits mutating loaded candidates; 5/5 |
| 5.3 | `tests/ocr/review.test.ts` | Unit | N/A (new file) | Tests preceded `indexing.ts` | 5/5 | New/reusable true/false, both CAS races, rejected no-index | Named CAS error mapping; 5/5 |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test | `NODE_ENV=test npx --no-install tsx --test tests/ocr/review.test.ts` — exit 0; 5 passed, 0 failed. |
| Runtime harness | Same runner with `--test-name-pattern "review HTTP rejects"` — exit 0; 1 passed; real localhost Express requests returned authenticated boundary outcomes without external services. |
| Rollback boundary | Remove `src/modules/ocr/{review,indexing}.ts` and `tests/ocr/review.test.ts`; revert only the Unit 9 route additions in `src/app.ts` and Unit 9 metadata. Preserve Units 1–8. |

### Validation and Boundary
- Canonical `NODE_ENV=test npm test` passed 65/65; `npm run check`, `npm run build`, tracked whitespace, and no-index whitespace checks passed.
- Unit 9 adds 341 functional changed lines and 38 metadata changed lines, 379 total, within the 400-line budget. No live PostgreSQL, OpenAI, Qdrant, or OCR process was used; no test process remained.
- Start: Unit 8 dispatch/status is complete. End: review/correction and indexing/CAS ports plus authenticated routes are green. Unit 10 task 5.4 remains unchecked.

## Unit 10: Rejection and Review UI
- Added authenticated rejection with current candidate-hash and state guards, required audit fields, durable store completion before success, and no indexing or activation call.
- Extended the existing static playground with authenticated candidate loading, protected page images, native/raw/candidate comparisons, line confidence/boxes, corrections, risks, approval, and rejection controls.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 5.4 | `tests/ocr/review.test.ts` | HTTP integration/static HTTP | Existing review suite passed 5/5 before edits. | Genuine RED passed only 5/7: rejection returned 404 instead of authenticated 401 and the playground lacked the review contract. A malformed-body edge then returned 409 instead of 400. | Focused suite passed 7/7 after minimal rejection and UI implementation. | Unauthorized, malformed, stale-hash, successful, repeated-state, durable-status, and static audit-field paths exercise distinct outcomes. | Shared UI decision-state helper extracted; focused suite remained 7/7. |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/ocr/review.test.ts` — exit 0; 7 passed, 0 failed. |
| Runtime harness command/scenario and exact result | `NODE_ENV=test npx --no-install tsx --test --test-name-pattern "authenticated rejection" tests/ocr/review.test.ts` — exit 0; 1 passed, 0 failed. Real localhost Express requests returned 401/400/409/200, persisted the rejected audit state through the injected store, retrieved rejected status, and made zero indexing calls. |
| Rollback boundary | Revert Unit 10 deltas in `src/app.ts`, `src/modules/ocr/review.ts`, `tests/ocr/review.test.ts`, and `public/playground/{index.html,app.js,styles.css}` plus Unit 10 metadata; preserve Units 1–9. |

### Validation and Boundary
- Canonical `NODE_ENV=test npm test` passed 67/67; `npm run check` and `npm run build` exited 0.
- Previously observed tracked `git diff --check` exited 0; no-index whitespace checks for Unit 10's untracked `src/modules/ocr/review.ts` and `tests/ocr/review.test.ts` exited 0.
- Unit 10 adds 271 functional and 42 metadata changed lines, 313 total, below the 400-line budget. No live database, embedding, vector, OCR, or external service was used; no persistent process was started.
- Start: Unit 9 review/indexing ports and approval routes are green. End: rejection isolation and the static review UI are green. Retention, deploy, OpenAPI, and E2E tasks remain untouched.
- No specification or design deviation.

## Unit 11: Retention and Flag-Off Isolation

### Implementation Summary
- Added a retention boundary that expires 30-day reviews before deletion, retains failed/rejected artifacts for 7 days, and retains superseded OCR artifacts for 30 days.
- Added exact state/artifact CAS claims, active-pointer checks, activation exclusion after a deletion claim, and restart-safe `retention_deleting→retention_deleted` completion.
- Wired retention into the existing exclusive reconciler cycle and made status/review/decision APIs return `404` without store access while OCR is disabled; native synchronous ingestion remains unchanged.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 6.1 | `tests/ocr/retention.test.ts` | Repository/HTTP/filesystem integration | Existing repository, dispatcher, and review suites passed 22/22. | Genuine missing-module RED failed before `retention.ts`; activation-race RED then failed 0/1 with “Missing expected rejection.” | Focused suite passed 6/6. | Exact TTL SQL, active pointer/state guards, flag-off `201/404`, and deletion-claim activation conflict exercise independent paths. | No production refactor was needed; two test-side expectations were corrected during GREEN without weakening behavior. |
| 6.2 | `tests/ocr/retention.test.ts` | Filesystem/reconciler integration | Same 22/22 safety net. | The missing-module RED preceded all production retention code. | Focused suite passed 6/6; relevant regression set passed 28/28. | Review expiry, terminal deletion, active preservation, simulated restart, repeat no-op, and exclusive reconciler execution cover distinct paths. | Named store boundary keeps filesystem behavior isolated; final focused suite remained 6/6. |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/ocr/retention.test.ts` — exit 0; 6 passed, 0 failed. |
| Runtime harness command/scenario and exact result | The focused suite used real temporary directories and localhost Express requests: interrupted deletion removed only the claimed directory, resumed to `retention_deleted`, repeated as a no-op, preserved the active directory, returned native `201`, and returned `404` for all disabled OCR candidate APIs without store access. |
| Rollback boundary | Remove `src/modules/ocr/retention.ts` and `tests/ocr/retention.test.ts`; revert only Unit 11 seams in `src/app.ts`, catalog repository/reconciler, and explicit enabled-state setup in dispatcher/review tests. Preserve Units 1–10. |

### Validation and Boundary
- Relevant regression suites passed 28/28 after explicitly enabling OCR in pre-existing enabled-path tests; the initial 19/22 run exposed only that test-fixture assumption.
- Canonical `NODE_ENV=test npm test` passed 73/73; `npm run check`, `npm run build`, tracked whitespace, and no-index whitespace checks for all seven intended untracked files passed.
- Temporary `rag-retention-*` directories were removed and no `tsx --test` or Node test process remained. No live database, OCR, embedding, vector, or external service was used; `ocr-service/.venv` was preserved.
- Unit 11 adds 277 functional changed lines and 50 metadata changed lines, 327 total, below the 400-line budget.
- Start: Unit 10 rejection/UI is complete. End: tasks 6.1–6.2 are complete. OpenAPI, deployment/configuration, E2E, production, commit, push, and PR work remain untouched.
- No specification or design deviation.

## Unit 12: Contracts and Deployment Wiring

### Implementation Summary
- Documented OCR `202` ingestion variants and authenticated status, review, correction/approval, and rejection routes with behavior-matched success and error responses.
- Added progress, review evidence, optimistic correction, decision, and structured error schemas while preserving the legacy-disabled `202` variants.
- Added private durable artifact defaults and OCR limits/timeouts; the root image now declares `/data/ingestions`, runs as `node`, and applies catalog migrations before startup without embedding secrets.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 6.3 | `tests/ocr/contracts-deploy.test.ts` | OpenAPI contract | Existing lifecycle/dispatcher/review/retention suites passed 42/42. | Genuine initial RED was 0/3: OCR `202` referenced the native schema and OCR routes/schemas were absent. | Focused suite passed 3/3. | A second RED required legacy-disabled `202` variants and accurate help authentication; final suite remained 3/3. | Shared schema references kept nested contracts reviewable; no behavior changed. |
| 6.4 | `tests/ocr/contracts-deploy.test.ts` | Config/static container | Same 42/42 safety net. | Initial RED found the old relative artifact root, absent limits/timeouts, volume, non-root user, and migration-aware command. | Focused suite passed 3/3; sanitized Docker build and image inspection passed. | A second RED required explicit runtime artifact-root wiring; final suite remained 3/3. | No further refactor needed. |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test command and exact result | `NODE_ENV=test npx --no-install tsx --test tests/ocr/contracts-deploy.test.ts` — exit 0; 3 passed, 0 failed. |
| Relevant regression command and exact result | Lifecycle, dispatcher, review, retention, and contract suites — exit 0; 45 passed, 0 failed. |
| Runtime harness | N/A — Unit 12 is static-only by task forecast. A sanitized-context Docker build succeeded and inspection proved user `node`, durable volume, production artifact root, and migration-before-server command. |
| Rollback boundary | Remove `tests/ocr/contracts-deploy.test.ts` and revert only Unit 12 deltas in `src/api/openapi.ts`, `src/config/env.ts`, and `Dockerfile`; preserve Units 1–11. |

### Validation and Boundary
- Canonical `NODE_ENV=test npm test` passed 76/76; `npm run check`, `npm run build`, tracked whitespace, and all intended-untracked whitespace checks passed.
- Docker build `rag-service:unit12-contract` succeeded from a sanitized 430.31 kB context; inspection matched the deployment contract. The image and temporary context were removed, no test process remained, and `ocr-service/.venv` was preserved.
- Unit 12 adds 219 functional lines and 50 metadata lines, 269 total, below the 400-line budget.
- Start: Unit 11 retention is complete. End: tasks 6.3–6.4 are complete. Unit 13 / all 7.x E2E and production work remains untouched.
- `docs/CONTRATO_CICLO_VIDA_Y_OCR.md` was read as authoritative and remained unchanged. No specification or design deviation.

## Unit 13: Local Deterministic E2E and Complete Gate

### Implementation Summary
- Added a bounded localhost HTTP suite covering native `201`, scanned `202 → review → approve → active`, and exact mixed native/OCR page reporting.
- Added resend identity reuse, fail-closed OCR exhaustion, and catalog-unavailable `503` coverage without external services.
- Made no production-code changes; the new scenarios passed immediately as baseline/characterization evidence, so no RED was fabricated.

### TDD Cycle Evidence
| Task | Test File | Layer | Safety Net | RED / Baseline | GREEN | TRIANGULATE | REFACTOR |
|---|---|---|---|---|---|---|---|
| 7.1 | `tests/ocr/e2e.test.ts` | Local HTTP E2E | Existing OCR suites passed 28/28. | Baseline/characterization: the test was written first and passed 2/2 immediately; no implementation gap existed at the local fake boundary. | Focused suite passed 2/2. | Native, scanned, corrected approval, post-activation status, and mixed page methods exercise distinct paths. | Test-only state naming improved; focused suite remained 2/2. |
| 7.2 | `tests/ocr/e2e.test.ts` | Local HTTP/dispatcher E2E | Same 28/28 safety net. | Baseline/characterization: resend, OCR-down, and catalog-down assertions passed immediately; no production code was written. | Focused suite passed 2/2. | Duplicate resend, terminal dispatch failure, status `503`, and retrieval `503` cover independent outcomes. | None needed beyond the shared test harness. |
| 7.3 | Gate commands below | Complete gate | Focused E2E passed 2/2. | N/A — verification-only task with no production behavior to implement. | All required gates passed. | Node type/build/test and offline Python tests provide independent stack evidence. | N/A — no production change. |

### Work Unit Evidence
| Evidence | Result |
|---|---|
| Focused test | `NODE_ENV=test npx --no-install tsx --test tests/ocr/e2e.test.ts` — exit 0; 2 passed, 0 failed. |
| Runtime harness | The focused suite started bounded ephemeral localhost Express servers and exercised real HTTP requests through ingest, status, review, approval, and retrieval routes; all servers closed through test cleanup. |
| Complete gate | `npm run check`, `npm run build`, and `NODE_ENV=test npm test` exited 0; canonical Node passed 78/78. `PYTHONDONTWRITEBYTECODE=1 ocr-service/.venv/bin/python -m pytest ocr-service/tests -q` passed 13/13 offline with one dependency deprecation warning. |
| Rollback boundary | Remove `tests/ocr/e2e.test.ts` and revert only tasks 7.1–7.3 plus this Unit 13 progress/history metadata; preserve Units 1–12. |

### Validation and Boundary
- Unit 13 adds 183 functional lines. Required tasks/progress/history metadata adds 52 changed lines, for 235 total, below the 400-line budget.
- Tracked and complete intended-untracked whitespace checks passed. No matching `tsx --test`, E2E Node, or OCR Uvicorn process remained after the corrected self-excluding process check.
- Intended untracked inventory: `src/modules/ocr/{dispatcher,indexing,retention,review}.ts`; `tests/ocr/{contracts-deploy,dispatcher,e2e,retention,review}.test.ts`.
- Tasks 7.1–7.3 are complete. Task 7.4 remains unchecked and untouched; no production flag, service, database, vector store, embedding provider, OCR endpoint, commit, push, or PR was used.