rag-service/openspec/changes/ocr-ingest-integration/apply-progress.md

Apply Progress: OCR Ingest Integration

Current State

• Mode: Strict TDD
• Delivery: Feature-branch-chain; Units 1–13 implemented as bounded slices; production acceptance remains pending
• Completed tasks: 1.1–1.4, 2.1–2.5, 3.1–3.4, 4.1–4.5, 5.1–5.4, 6.1–6.4, 7.1–7.3
• Overall task progress: 29/30 complete

Unit 1: Migration

Implementation Summary

• Added migrations/002_ocr_review.sql with durable OCR jobs, auditable page records, atomic review correction targets, and a partial unique index for non-terminal OCR identities.
• Added focused schema contract coverage in tests/catalog/migration-002.test.ts.
• Preserved migrations/001_knowledge_lifecycle.sql and all repository logic unchanged.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
1.2	tests/catalog/migration-002.test.ts	Schema contract	N/A (new files)	Valid RED: 4/4 failed with ENOENT because migration 002 did not exist. An earlier syntax-invalid run was corrected and is not counted as RED.	4/4 passed with migration 002 present.	4 independent schema behaviors cover jobs, pages, corrections, and pending identity uniqueness.	None needed; SQL already matches the minimal authoritative contract.

Test Summary

• Total tests written: 4
• Total tests passing: 4
• Layers used: Schema contract (4)
• Approval tests: None — no existing production file was modified
• Pure functions created: 0

Work Unit Evidence

Evidence	Result
Focused test command and exact result	npx --no-install tsx --test tests/catalog/migration-002.test.ts — exit 0; 4 tests, 4 passed, 0 failed.
Runtime harness command/scenario and exact result	N/A — the task forecast defines Unit 1 as schema-only, and the project has no PostgreSQL test harness or database-test dependency. Applying a live migration would require external database access outside this unit.
Rollback boundary	Delete migrations/002_ocr_review.sql and tests/catalog/migration-002.test.ts, revert task 1.2 to pending, and remove this Unit 1 progress/history entry. No existing migration or repository behavior must be reverted.

Additional Validation

• npm run check — exit 0.
• git diff --check — exit 0 after removing one trailing-space warning introduced in the history header.

Deviations

None — the migration follows the proposal, specifications, design, and closed OCR persistence contract.

Remaining Persistence Tasks

• 1.1 Complete repository-level RED coverage for duplicate pending ingestion, lease recovery, and review transitions in Unit 2.
• 1.2 Add migration 002 schema and pending OCR identity unique index.
• 1.3 Add catalog repository candidates and leases in Unit 2.
• 1.4 Refactor persistence code after Unit 2 reaches green.

Review Boundary

• Start: Migration 001 already supplies lifecycle states and parent catalog tables.
• End: Migration 002 and its focused schema contract test are green.
• Out of scope: Repository methods, dispatch behavior, runtime PostgreSQL migration, Unit 2, commits, pushes, PRs, deployment, and secrets.
• Rollback: Remove only the two Unit 1 implementation files and their progress metadata.
• Authored review budget: 225 additions/deletions, below the 400-line budget for this autonomous slice.

Unit 2: Repository

Implementation Summary

• Added pending OCR identity lookup, atomic queue claiming, expired-lease recovery, and guarded review transitions to CatalogRepository.
• Added a focused fake-pool repository harness that covers duplicate reuse, empty queues, remote identity preservation, and invalid transitions.
• Corrected the canonical test script so it executes both root-level and nested test suites.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
1.1	tests/catalog/repository-ocr.test.ts	Unit/fake pool	4/4 existing tests passed	0/6 with methods absent; then 5/6 while each completion gate was missing	6/6 passed after task 1.3	Found/not-found candidates, claimed/empty queue, expired lease, valid/invalid transition	Covered by 1.4
1.3	tests/catalog/repository-ocr.test.ts	Unit/fake pool	4/4 existing tests passed	Tests preceded production code	6/6 passed	Multiple inputs and state paths exercised	Shared OCR row mapping extracted only after green
1.4	tests/catalog/repository-ocr.test.ts	Approval refactor	6/6 before refactor	N/A — behavior-preserving approval baseline	6/6 after refactor	Existing six cases preserved	Reusable aliased-column projection removed SQL duplication

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/catalog/repository-ocr.test.ts — exit 0; 6 tests passed, 0 failed. NODE_ENV=test npx --no-install tsx --test tests/catalog/*.test.ts — exit 0; 10 tests passed, 0 failed.
Runtime harness command/scenario and exact result	NODE_ENV=test npx --no-install tsx --test tests/*.test.ts — exit 0; 25 root tests passed, 0 failed. Canonical npm test — exit 0; 35 total tests passed, 0 failed, proving the 25 root and 10 nested tests execute together.
Rollback boundary	Revert Unit 2 changes in src/modules/catalog/repository.ts, delete tests/catalog/repository-ocr.test.ts, and revert the package.json test script; Unit 1 migration remains intact.

Validation and Boundary

• Corrective rerun for failed evidence revision sha256:9433c0cd6628f36f4c96a27d41c6d429f4303d038ffe928f297367c274f21c2f; fresh evidence revision sha256:a19119f84381447f54a6223e2a203acaf9d3ae08b6ae8f585e3bd52cd510909b.
• The failed canonical script expanded only nested tests after nested suites appeared. The minimal correction is NODE_ENV=test tsx --test tests/*.test.ts tests/**/*.test.ts.
• Focused repository suite passed 6/6; all catalog suites passed 10/10; explicit root suites passed 25/25; canonical npm test passed 35/35; npm run check and git diff --check exited 0.
• Native Unit 2 accounting was 511 changed lines. The maintainer approved size:exception exclusively for Unit 2; the corrective attempt has a separate 600-line remediation budget. All later units retain the normal 400-line budget.
• No design deviations. No Unit 3 work was started.

Corrective TDD Evidence

Correction	Safety Net / RED	GREEN	TRIANGULATE	REFACTOR
Canonical test discovery	Failed evidence revision proved canonical npm test executed only 10 nested tests while the explicit root command executed 25 tests.	Updated only the package test argument list; canonical npm test passed 35/35.	Direct root 25/25 and nested catalog 10/10 results sum to and identify the canonical 35/35 run.	None needed — one package-script argument was added.

Unit 3: Extraction

Implementation Summary

• Added a Node 22 fixture spike proving pdf-parse page callbacks backed by bundled PDF.js 2.0.550 preserve ordered pages, including a blank page.
• Added parsePdfPages with one-based page identity, raw native text, stable SHA-256 hashes, and fail-closed complete-page accounting.
• Preserved textual ingestion as data-only, excluded CMakeLists.txt, MDX, and shell lookalikes, and kept task 2.2 pending for Unit 4 detection/composition assertions.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
2.1	tests/parsers/pdf-pages.test.ts; scripts/spike-pdfjs.ts	Runtime spike	20/20 lifecycle service tests passed	Focused suite failed 0/1 because parsePdfPages did not exist; the spike and parser implementation were still absent.	Node 22.14.0 spike returned pages 1–3 exactly, including blank page 2.	Three pages prove non-empty, blank, and later-page ordering paths.	Replaced the over-budget direct dependency with the design-approved existing pdf-parse pagerender path; spike remained green.
2.3	tests/parsers/pdf-pages.test.ts	Unit/integration fixture	20/20 lifecycle service tests passed	Focused suite failed 0/1 on the missing export before production code.	Final focused run passed 5/5 after implementation.	Covered page hashes/blank page, native PDF composition, non-PDF rejection, data-only text parsing, and unsupported lookalikes.	Focused suite remained 5/5 after consolidating on the existing parser dependency.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/parsers/pdf-pages.test.ts — exit 0; 5 passed, 0 failed.
Runtime harness command/scenario and exact result	NODE_ENV=test npx --no-install tsx scripts/spike-pdfjs.ts — exit 0 on Node 22.14.0; PDF.js 2.0.550 returned three ordered pages with page 2 blank.
Rollback boundary	Revert src/modules/parsers/parser-registry.ts; delete scripts/spike-pdfjs.ts, tests/parsers/pdf-pages.test.ts, and tests/fixtures/ocr/native-three-pages.pdf; revert tasks 2.1/2.3 and this Unit 3 progress/history entry. Units 1–2 remain intact.

Validation and Boundary

• Canonical npm test passed 40/40; npm run check and git diff --check exited 0.
• Native authored slice before progress/history metadata: 225 additions/deletions; no package or lockfile delta remains.
• Start: Unit 2 persistence is complete. End: per-page native extraction and parser threat routing are green.
• Out of scope: detection thresholds, OCR selection, composition, risks, Unit 4 tasks 2.4/2.5, commits, pushes, PRs, deployment, and restricted paths.
• No specification deviation. The design-authorized pdf-parse pagerender fallback was selected only after its Node 22 fixture spike passed.

Unit 4: Detection and Composition

Implementation Summary

• Added pure per-page native detection under pdf-detection-v1, exact OCR page selection, verified-blank classification, and fail-closed OCR quality gates.
• Added deterministic candidate composition using exactly one source per page, ordered OCR lines, blank omission, canonical hashes, and unchanged risk tokens prioritized for review.
• Completed the remaining task 2.2 scenarios by combining the Unit 3 data-only/parser threat coverage with Unit 4 mixed-PDF, detection, blank, hash, and risk RED assertions.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
2.2	tests/ocr/detection.test.ts; tests/parsers/pdf-pages.test.ts	Unit/integration fixture	Unit 3 parser suite passed 5/5 before edits.	Focused Unit 4 suite failed before production with ERR_MODULE_NOT_FOUND for src/modules/ocr/detection.js.	Focused suite passed 5/5 after detection and composition were added.	Real generated mixed PDF selected pages 2 and 3; .txt/.md stayed data-only and CMake/MDX/shell lookalikes stayed unsupported.	None needed; routing remains separated between parser support and the pure PDF OCR planner.
2.4	tests/ocr/detection.test.ts	Unit	N/A (new production file)	Threshold, mixed-page, blank, and quality assertions preceded production code.	Exact native, blank, and OCR quality boundaries passed in the 5/5 focused suite.	Every native and OCR threshold has a passing boundary plus a failing neighbor; visible ink with empty OCR is blocked.	Pure functions and named policy version were present at green; final focused suite remained 5/5.
2.5	tests/ocr/detection.test.ts	Unit	N/A (new production file)	Ordered composition, blank omission, canonical hash, audit-source, and risk assertions preceded production code.	Deterministic composition and risk prioritization passed in the 5/5 focused suite.	Reordered pages, tied OCR boxes, blank middle page, repeated token, and four known ambiguous tokens exercise distinct paths.	Reused shared canonical JSON and SHA-256 utilities; final focused suite and type check remained green.

Test Summary

• Tests: 5 written and focused-passing; 45 canonical (unit 4, integration fixture 1)
• Approval tests: None — new production files; pure functions created: 6

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/ocr/detection.test.ts — exit 0; 5 tests passed, 0 failed.
Runtime harness command/scenario and exact result	NODE_ENV=test npx --no-install tsx --test --test-name-pattern "parsed mixed PDF" tests/ocr/detection.test.ts — exit 0; 1 test passed, 0 failed. The harness generated a three-page PDF, parsed it through the Node 22 pdf-parse page callback, and selected only blank page 2 and insufficient page 3 for OCR.
Rollback boundary	Delete src/modules/ocr/detection.ts, src/modules/ocr/composition.ts, and tests/ocr/detection.test.ts; revert tasks 2.2/2.4/2.5 and this Unit 4 progress/history entry. Units 1–3 and their parser threat coverage remain intact.

Validation and Boundary

• Canonical npm test passed 45/45; npm run check, tracked git diff --check, and no-index whitespace checks for all three new files exited 0.
• Authored Unit 4 code and tests total 322 additions and 0 deletions, below the 400-line budget with no size exception.
• Start: Unit 3 page extraction and parser threat routing are complete. End: detection, blank/quality gates, deterministic composition, hashes, and risk tokens are green.
• Out of scope: OCR service, client, dispatcher, ingest routing, review APIs, commits, pushes, PRs, deployment, and restricted paths.
• No specification or design deviation.

Unit 5: OCR Service

Implementation and Correction Summary

• Added pinned Unit 5 FastAPI test dependencies and retained the ignored ocr-service/.venv environment.
• Added RED-first HTTP tests and a SQLite-backed private API for bearer auth, idempotency, allowlisting, upload/page limits, queue pressure, status, deletion, and health.
• Preserved failed evidence sha256:4a5c8ed8d8b302f7ff654d4f40fe0f13049bfdd29db4c76576f3847c0a9be15a: its harness returned 401/400/400; inspection showed & backgrounded the preceding shell AND-list, so request variables existed only in the child shell and authenticated curls sent an empty request form, reproduced as 400 INVALID_REQUEST.
• Corrected only the harness command boundary; the 381-line candidate implementation remained unchanged before progress persistence.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
3.1	ocr-service/tests/test_api.py	HTTP integration	N/A (new service)	Collection failed with ModuleNotFoundError: No module named 'app'; failed localhost runtime evidence then supplied correction RED.	8/8 focused tests and corrected curl 401/202/409 passed.	Auth variants, idempotency/conflict, allowlist, limits, pressure, integrity, and real multipart requests exercised distinct paths.	No production refactor: the defect was shell variable scope in the harness.
3.2	ocr-service/tests/test_api.py	HTTP integration/SQLite	N/A (new service)	Same missing-service RED covered the API; failed harness blocked completion.	8/8 focused tests and real Uvicorn harness passed.	Ready/not-ready health, queue capacity, status, deletion, and process cleanup exercised distinct paths.	Pure page hashing and centralized terminal errors retained; no correction needed.

Work Unit Evidence

• Focused: ocr-service/.venv/bin/python -m pytest ocr-service/tests -k auth — exit 0; 8 passed, 0 failed (one dependency deprecation warning).
• Runtime: local Uvicorn plus multipart curl — unauthorized 401 with UNAUTHORIZED, accepted 202 with queued identity, conflicting same key 409 with IDEMPOTENCY_CONFLICT.
• Gates: npm test 45/45, npm run check, Python compileall, tracked and untracked whitespace checks — all exit 0/equivalent clean.
• Cleanup: Uvicorn PID 268425 terminated and absent; temporary PDF, SQLite DB, and log removed; .venv retained.
• Rollback: remove .gitignore Unit 5 entries and ocr-service/, then revert tasks 3.1/3.2 and this Unit 5 progress/history block; Units 1–4 remain intact.
• Fresh evidence revision: sha256:0d8b2c57bbab967473eaf99a7ca900168554e64254fa80197b3626b4570b58d0; corrected final authored change count: 397 additions plus deletions, within 400.

Unit 6: OCR Rendering and Image

Implementation Summary

• Added deterministic 200 DPI PDF page rendering with a pre-render 25-megapixel guard, a PaddleOCR adapter, stable line identities, quality metrics, and the exact result schema.
• Added a deterministic fake-driven test seam and preserved ambiguous OCR tokens without correction.
• Added the private CPU image with PaddleOCR 3.4.0, PaddlePaddle CPU 3.2.2, baked Latin/mobile models, one Uvicorn worker, a non-root user, private job storage, healthcheck, and documented 3 CPU/5 GiB deployment limits.
• Added a Dockerfile-specific deny-by-default build context so restricted and unrelated repository paths are never sent to the Docker daemon.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
3.3	ocr-service/tests/test_render.py	Unit/integration fixture	Existing OCR API suite passed 8/8 before production edits.	Focused collection failed because app.engine did not exist.	Focused render suite passed 5/5; complete OCR suite passed 13/13.	Selected pages 1/3, invalid page and pixel-limit paths, repeatable fake results, and Paddle payload normalization exercise distinct behavior.	Named constants and pure metric/result transforms retained; focused suite remained 5/5.
3.4	ocr-service/tests/test_render.py	Static/container runtime	Existing OCR API suite passed 8/8 before production edits.	Docker contract was absent; deny-by-default context triangulation then failed until Dockerfile.dockerignore existed.	Image built with baked models; offline constrained container rendered and validated PNG/result output.	Static pin/runtime assertions plus real model build and --network none execution cover configuration and runtime paths.	Added missing OpenCV runtime libraries after the first build exposed libGL.so.1; exact corrected image passed.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	ocr-service/.venv/bin/python -m pytest ocr-service/tests -k render -q — exit 0; 5 passed, 8 deselected, 1 dependency deprecation warning.
Runtime harness command/scenario and exact result	docker build --file ocr-service/Dockerfile --tag rag-ocr-service:unit6 . — corrected exact image exited 0 and baked both models. A docker run --rm --network none --cpus 3 --memory 5g ... harness loaded cached PaddleOCR 3.4.0/PaddlePaddle 3.2.2 models, rendered a 1700x2200 PNG (23,309 bytes), returned schema 1, one page, and one OCR line; host Pillow reopened it as PNG 1700x2200.
Rollback boundary	Remove ocr-service/{Dockerfile,Dockerfile.dockerignore,app/engine.py,app/models.py,app/render.py,tests/test_render.py} and revert Unit 6 changes in app/main.py, requirements.txt, and README.md; Units 1–5 remain intact.

Validation and Boundary

• RED was followed by GREEN and post-refactor reruns. One expected GREEN iteration corrected a test-side non-whitespace count from 16 to the mathematically correct 15.
• The first image build failed on missing libGL.so.1; adding the required slim-image runtime libraries produced a successful build and runtime harness. A later optional connectivity-check refactor rebuild exhausted Docker storage; that unvalidated one-line refactor was reverted, the validated Dockerfile bytes were restored, and the image tag was removed during cleanup.
• npm test passed 45/45; npm run check, Python compileall, tracked/untracked whitespace checks, and the full OCR pytest suite (13/13) passed.
• Start: Unit 5 private queue/API is complete. End: Unit 6 render/engine/schema/container behavior is complete. Unit 7 client work was not started.
• Native Unit 6 implementation/tests/docs are 397 authored additions plus deletions, within the 400-line budget; required SDD progress and workspace history metadata are administrative evidence outside that native slice.
• No specification or design deviation. The Docker image is private by deployment contract; CPU/RAM labels document limits that the platform must enforce.

Unit 7: OCR Client and Durable Artifacts

Implementation Summary

• Added a private OCR HTTP client that preserves the exact idempotency key across one initial submission and at most two transient resubmissions, treats 429 as retryable queue pressure without immediate resubmission, and treats deterministic failures as terminal.
• Added 2–15 second polling backoff plus strict acknowledgement, status, engine, job, document, ordered-page, metrics, line, and schema validation before a result can be persisted.
• Added durable 0600 original and canonical manifest writes, UUIDv5 document artifact identities, lexical path containment, verifiable hashes, and age-gated orphan sweeping that preserves retained, recent, non-UUID, and symlink entries.
• Left task 4.1 pending because Unit 7 proves only its retries/integrity clause; Unit 8 still owns flag-off 201, OCR 202/status, OCR-down fail-closed, and catalog-down 503 HTTP coverage.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
4.2	tests/ocr/client.test.ts	Unit/runtime fetch stub	Existing OCR detection suite passed 5/5; production file was new.	Focused suite failed with ERR_MODULE_NOT_FOUND for src/modules/ocr/client.js before production code existed.	Final focused suite passed 6/6; runtime retry scenario passed 1/1.	Network then 503 produced exactly three same-key attempts with 2s/4s backoff; 429 and 422 each stopped after one attempt; malformed acknowledgement/schema/pages failed integrity; polling used 2s/4s.	Adapted multipart bytes to a type-safe Uint8Array; focused suite and type check remained green.
4.3	tests/ocr/client.test.ts	Filesystem integration	Existing OCR detection suite passed 5/5; production file was new.	The same missing-module RED preceded artifact production code.	Private originals, canonical manifest, safe resolution, and sweep passed in the final 6/6 suite.	Two out-of-order documents proved canonical ordering/hashes; traversal was rejected; sweep removed only an old orphan while preserving retained/recent directories and a symlink.	A test-side assertion was corrected to inspect the retained dangling symlink with lstat; no production behavior changed, and the suite passed 6/6.

Test Summary

• Tests written and passing: 6 focused tests; unit/fetch-stub and filesystem integration layers.
• Approval tests: None — both production modules are new.
• RED: exit 1, module-not-found before either production file existed.
• GREEN/REFACTOR: exit 0, 6 passed, 0 failed.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/ocr/client.test.ts — exit 0; 6 passed, 0 failed.
Runtime harness command/scenario and exact result	NODE_ENV=test npx --no-install tsx --test --test-name-pattern "runtime retry stub" tests/ocr/client.test.ts — exit 0; 1 passed, 0 failed. Deterministic stub observed network failure, 503, then 202: exactly 3 submission attempts, one unchanged idempotency key, 2,000/4,000 ms backoffs, followed by a strictly validated two-page result.
Rollback boundary	Remove src/modules/ocr/client.ts, src/modules/ocr/artifacts.ts, and tests/ocr/client.test.ts; revert tasks 4.2/4.3 and this Unit 7 progress/history metadata. Units 1–6 remain intact.

Validation and Boundary

• npm test passed 51/51; npm run check, npm run build, tracked whitespace, and no-index whitespace checks for all three new files passed.
• Test-created rag-ocr-artifacts-* and rag-ocr-sweep-* directories were removed; no server, external OCR call, or persistent process was started; ocr-service/.venv was preserved.
• Start: Unit 6 OCR rendering and container runtime are complete. End: Unit 7 client retries/integrity and durable artifacts/orphan sweep are implemented and green.
• Out of scope and untouched: dispatcher, reconciler, ingest branch, app upload/status routes, Unit 8 tests, commits, pushes, PRs, native review, deployment, and restricted paths.
• The implementation/test slice is 506 authored additions (client 174, artifacts 129, tests 203). This exceeds the 400-line default after one honest cohesive assessment; no code-golf or second slicing pass was attempted. The maintainer explicitly approved size:exception, including 585 total native-accounting lines with required metadata.
• No specification or design deviation.

Approved Size-Exception Corrective Context

• The native attempt was reset after failed evidence revision sha256:dd224c3864adc8380dc7d9fd147cb448f5da8138277f3f346a35d7eeb3aca158; the failure was budget-only, not a functional defect.
• Corrective work unit unit-7-size-exception-validation uses attempt token sha256:0109cfd8d3b37a5672c65a37700269b89cb80e4e357a39f68a920779a1715c39 and must settle against the failed revision through the parent.
• Production and test files remain unchanged for the corrective rerun; only approval/reset evidence and fresh validation metadata may change.
• Task 4.1 remains pending, while tasks 4.2 and 4.3 remain complete. Unit 8 was not started.

Unit 8: Dispatcher and Runtime Orchestration

• Completed multi-document progress, uniqueness-race reuse, exact lease recovery, persisted-key dispatch, default runtime wiring, authenticated status, and fail-closed retrieval; no activation or embeddings occur on OCR failure.

Evidence Chain and Review Accounting

• Complete review slice: Relative to committed HEAD ac046e3, Unit 8 contains 1,045 functional changed lines plus 41 required metadata lines, for 1,086 total changed lines. The complete slice is not within 400 lines, and no size:exception was approved or recorded for it.
• Interrupted preserved candidate: Native failed/interrupted revision sha256:182cc73954a8518d03bd6c3ef5f4141b352aa9b69fca4177447d76deaddad223 contained 815 changed lines.
• Auto-chain reset: The maintainer-selected preflight strategy was auto-chain with feature-branch-chain. Native reset revision sha256:c0db82cf3db8ae790d34e4c310b998f97bb36864c97c3aa84e4c87ab77d07c2a preserved the interrupted candidate and scoped the next bounded recovery objective only to additional changes; it did not approve the complete Unit 8 slice as a size exception.
• Recovery delta: After that reset, the bounded recovery added 350 functional changed lines plus 41 required metadata lines, for 391 total changed lines within the separate 400-line recovery objective.
• Passed recovery: Fresh passed evidence revision sha256:e424afe7b2f8efd547dcfd0baf64ec1b46ac344f39afd87259560a094d59163f closes the interrupted → auto-chain reset → passed recovery chain.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
4.1	tests/ocr/{dispatcher,client}.test.ts	HTTP/integration	Recovered baseline 7/7 passed; not labeled RED	New scenarios produced genuine 5/9 RED; catalog /retrieve already failed closed	Unit 8 9/9; client 6/6	201/202/status, OCR failure, 503 retrieval, retries/integrity	Consolidated persisted-key assertion; 9/9
4.4	tests/ocr/dispatcher.test.ts	Unit/integration	Recovered baseline 7/7	Exact recovery failed by claiming unrelated work	9/9	Live/new queue separation plus periodic dispatch	Exact-job dispatch helper; 9/9
4.5	tests/ocr/dispatcher.test.ts	HTTP/filesystem	Recovered baseline 7/7	Runtime wiring RED was 8/10	9/9	Multipart upload, durable reload, duplicate race, mixed documents	Shared queue drain; 9/9

Work Unit Evidence

Evidence	Result
Focused test	NODE_ENV=test npx --no-install tsx --test tests/ocr/dispatcher.test.ts — 9 passed, 0 failed.
Focused repository OCR test	NODE_ENV=test npx --no-install tsx --test tests/catalog/repository-ocr.test.ts — 6 passed, 0 failed.
Canonical Node test	npm test — 60 passed, 0 failed.
Runtime harness	Bounded localhost curl — ingest 202/ocr_queued; authenticated status 200/ocr_queued; leaked child from initial shell cleanup was detected, terminated, and port/temp cleanup reverified.
Rollback boundary	Revert Unit 8 deltas in src/{app,config/env}.ts, catalog/ingest/client modules, src/modules/ocr/dispatcher.ts, and its test; preserve Units 1–7.

Unit 9: Review and Indexing Core

• Added authenticated review/approval routes with injectable service boundaries, immutable correction preparation, stale/duplicate conflict guards, and new/reusable activation CAS behavior. Rejection transition and playground UI remain Unit 10.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
5.1	tests/ocr/review.test.ts	HTTP/unit	App suites 9/9 and 20/20	Missing indexing.js; exit 1	5/5	Unauthorized/premature, candidate/line/active staleness, duplicate targets, new/reusable races, rejected no-index	Assertions remained 5/5
5.2	tests/ocr/review.test.ts	HTTP/unit	29/29	Tests preceded review.ts	5/5	Audit view plus two corrections and four conflict paths	Clone prevents failed commits mutating loaded candidates; 5/5
5.3	tests/ocr/review.test.ts	Unit	N/A (new file)	Tests preceded indexing.ts	5/5	New/reusable true/false, both CAS races, rejected no-index	Named CAS error mapping; 5/5

Work Unit Evidence

Evidence	Result
Focused test	NODE_ENV=test npx --no-install tsx --test tests/ocr/review.test.ts — exit 0; 5 passed, 0 failed.
Runtime harness	Same runner with --test-name-pattern "review HTTP rejects" — exit 0; 1 passed; real localhost Express requests returned authenticated boundary outcomes without external services.
Rollback boundary	Remove src/modules/ocr/{review,indexing}.ts and tests/ocr/review.test.ts; revert only the Unit 9 route additions in src/app.ts and Unit 9 metadata. Preserve Units 1–8.

Validation and Boundary

• Canonical NODE_ENV=test npm test passed 65/65; npm run check, npm run build, tracked whitespace, and no-index whitespace checks passed.
• Unit 9 adds 341 functional changed lines and 38 metadata changed lines, 379 total, within the 400-line budget. No live PostgreSQL, OpenAI, Qdrant, or OCR process was used; no test process remained.
• Start: Unit 8 dispatch/status is complete. End: review/correction and indexing/CAS ports plus authenticated routes are green. Unit 10 task 5.4 remains unchecked.

Unit 10: Rejection and Review UI

• Added authenticated rejection with current candidate-hash and state guards, required audit fields, durable store completion before success, and no indexing or activation call.
• Extended the existing static playground with authenticated candidate loading, protected page images, native/raw/candidate comparisons, line confidence/boxes, corrections, risks, approval, and rejection controls.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
5.4	tests/ocr/review.test.ts	HTTP integration/static HTTP	Existing review suite passed 5/5 before edits.	Genuine RED passed only 5/7: rejection returned 404 instead of authenticated 401 and the playground lacked the review contract. A malformed-body edge then returned 409 instead of 400.	Focused suite passed 7/7 after minimal rejection and UI implementation.	Unauthorized, malformed, stale-hash, successful, repeated-state, durable-status, and static audit-field paths exercise distinct outcomes.	Shared UI decision-state helper extracted; focused suite remained 7/7.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/ocr/review.test.ts — exit 0; 7 passed, 0 failed.
Runtime harness command/scenario and exact result	NODE_ENV=test npx --no-install tsx --test --test-name-pattern "authenticated rejection" tests/ocr/review.test.ts — exit 0; 1 passed, 0 failed. Real localhost Express requests returned 401/400/409/200, persisted the rejected audit state through the injected store, retrieved rejected status, and made zero indexing calls.
Rollback boundary	Revert Unit 10 deltas in src/app.ts, src/modules/ocr/review.ts, tests/ocr/review.test.ts, and public/playground/{index.html,app.js,styles.css} plus Unit 10 metadata; preserve Units 1–9.

Validation and Boundary

• Canonical NODE_ENV=test npm test passed 67/67; npm run check and npm run build exited 0.
• Previously observed tracked git diff --check exited 0; no-index whitespace checks for Unit 10's untracked src/modules/ocr/review.ts and tests/ocr/review.test.ts exited 0.
• Unit 10 adds 271 functional and 42 metadata changed lines, 313 total, below the 400-line budget. No live database, embedding, vector, OCR, or external service was used; no persistent process was started.
• Start: Unit 9 review/indexing ports and approval routes are green. End: rejection isolation and the static review UI are green. Retention, deploy, OpenAPI, and E2E tasks remain untouched.
• No specification or design deviation.

Unit 11: Retention and Flag-Off Isolation

Implementation Summary

• Added a retention boundary that expires 30-day reviews before deletion, retains failed/rejected artifacts for 7 days, and retains superseded OCR artifacts for 30 days.
• Added exact state/artifact CAS claims, active-pointer checks, activation exclusion after a deletion claim, and restart-safe retention_deleting→retention_deleted completion.
• Wired retention into the existing exclusive reconciler cycle and made status/review/decision APIs return 404 without store access while OCR is disabled; native synchronous ingestion remains unchanged.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
6.1	tests/ocr/retention.test.ts	Repository/HTTP/filesystem integration	Existing repository, dispatcher, and review suites passed 22/22.	Genuine missing-module RED failed before retention.ts; activation-race RED then failed 0/1 with “Missing expected rejection.”	Focused suite passed 6/6.	Exact TTL SQL, active pointer/state guards, flag-off 201/404, and deletion-claim activation conflict exercise independent paths.	No production refactor was needed; two test-side expectations were corrected during GREEN without weakening behavior.
6.2	tests/ocr/retention.test.ts	Filesystem/reconciler integration	Same 22/22 safety net.	The missing-module RED preceded all production retention code.	Focused suite passed 6/6; relevant regression set passed 28/28.	Review expiry, terminal deletion, active preservation, simulated restart, repeat no-op, and exclusive reconciler execution cover distinct paths.	Named store boundary keeps filesystem behavior isolated; final focused suite remained 6/6.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/ocr/retention.test.ts — exit 0; 6 passed, 0 failed.
Runtime harness command/scenario and exact result	The focused suite used real temporary directories and localhost Express requests: interrupted deletion removed only the claimed directory, resumed to retention_deleted, repeated as a no-op, preserved the active directory, returned native 201, and returned 404 for all disabled OCR candidate APIs without store access.
Rollback boundary	Remove src/modules/ocr/retention.ts and tests/ocr/retention.test.ts; revert only Unit 11 seams in src/app.ts, catalog repository/reconciler, and explicit enabled-state setup in dispatcher/review tests. Preserve Units 1–10.

Validation and Boundary

• Relevant regression suites passed 28/28 after explicitly enabling OCR in pre-existing enabled-path tests; the initial 19/22 run exposed only that test-fixture assumption.
• Canonical NODE_ENV=test npm test passed 73/73; npm run check, npm run build, tracked whitespace, and no-index whitespace checks for all seven intended untracked files passed.
• Temporary rag-retention-* directories were removed and no tsx --test or Node test process remained. No live database, OCR, embedding, vector, or external service was used; ocr-service/.venv was preserved.
• Unit 11 adds 277 functional changed lines and 50 metadata changed lines, 327 total, below the 400-line budget.
• Start: Unit 10 rejection/UI is complete. End: tasks 6.1–6.2 are complete. OpenAPI, deployment/configuration, E2E, production, commit, push, and PR work remain untouched.
• No specification or design deviation.

Unit 12: Contracts and Deployment Wiring

Implementation Summary

• Documented OCR 202 ingestion variants and authenticated status, review, correction/approval, and rejection routes with behavior-matched success and error responses.
• Added progress, review evidence, optimistic correction, decision, and structured error schemas while preserving the legacy-disabled 202 variants.
• Added private durable artifact defaults and OCR limits/timeouts; the root image now declares /data/ingestions, runs as node, and applies catalog migrations before startup without embedding secrets.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED	GREEN	TRIANGULATE	REFACTOR
6.3	tests/ocr/contracts-deploy.test.ts	OpenAPI contract	Existing lifecycle/dispatcher/review/retention suites passed 42/42.	Genuine initial RED was 0/3: OCR 202 referenced the native schema and OCR routes/schemas were absent.	Focused suite passed 3/3.	A second RED required legacy-disabled 202 variants and accurate help authentication; final suite remained 3/3.	Shared schema references kept nested contracts reviewable; no behavior changed.
6.4	tests/ocr/contracts-deploy.test.ts	Config/static container	Same 42/42 safety net.	Initial RED found the old relative artifact root, absent limits/timeouts, volume, non-root user, and migration-aware command.	Focused suite passed 3/3; sanitized Docker build and image inspection passed.	A second RED required explicit runtime artifact-root wiring; final suite remained 3/3.	No further refactor needed.

Work Unit Evidence

Evidence	Result
Focused test command and exact result	NODE_ENV=test npx --no-install tsx --test tests/ocr/contracts-deploy.test.ts — exit 0; 3 passed, 0 failed.
Relevant regression command and exact result	Lifecycle, dispatcher, review, retention, and contract suites — exit 0; 45 passed, 0 failed.
Runtime harness	N/A — Unit 12 is static-only by task forecast. A sanitized-context Docker build succeeded and inspection proved user node, durable volume, production artifact root, and migration-before-server command.
Rollback boundary	Remove tests/ocr/contracts-deploy.test.ts and revert only Unit 12 deltas in src/api/openapi.ts, src/config/env.ts, and Dockerfile; preserve Units 1–11.

Validation and Boundary

• Canonical NODE_ENV=test npm test passed 76/76; npm run check, npm run build, tracked whitespace, and all intended-untracked whitespace checks passed.
• Docker build rag-service:unit12-contract succeeded from a sanitized 430.31 kB context; inspection matched the deployment contract. The image and temporary context were removed, no test process remained, and ocr-service/.venv was preserved.
• Unit 12 adds 219 functional lines and 50 metadata lines, 269 total, below the 400-line budget.
• Start: Unit 11 retention is complete. End: tasks 6.3–6.4 are complete. Unit 13 / all 7.x E2E and production work remains untouched.
• docs/CONTRATO_CICLO_VIDA_Y_OCR.md was read as authoritative and remained unchanged. No specification or design deviation.

Unit 13: Local Deterministic E2E and Complete Gate

Implementation Summary

• Added a bounded localhost HTTP suite covering native 201, scanned 202 → review → approve → active, and exact mixed native/OCR page reporting.
• Added resend identity reuse, fail-closed OCR exhaustion, and catalog-unavailable 503 coverage without external services.
• Made no production-code changes; the new scenarios passed immediately as baseline/characterization evidence, so no RED was fabricated.

TDD Cycle Evidence

Task	Test File	Layer	Safety Net	RED / Baseline	GREEN	TRIANGULATE	REFACTOR
7.1	tests/ocr/e2e.test.ts	Local HTTP E2E	Existing OCR suites passed 28/28.	Baseline/characterization: the test was written first and passed 2/2 immediately; no implementation gap existed at the local fake boundary.	Focused suite passed 2/2.	Native, scanned, corrected approval, post-activation status, and mixed page methods exercise distinct paths.	Test-only state naming improved; focused suite remained 2/2.
7.2	tests/ocr/e2e.test.ts	Local HTTP/dispatcher E2E	Same 28/28 safety net.	Baseline/characterization: resend, OCR-down, and catalog-down assertions passed immediately; no production code was written.	Focused suite passed 2/2.	Duplicate resend, terminal dispatch failure, status 503, and retrieval 503 cover independent outcomes.	None needed beyond the shared test harness.
7.3	Gate commands below	Complete gate	Focused E2E passed 2/2.	N/A — verification-only task with no production behavior to implement.	All required gates passed.	Node type/build/test and offline Python tests provide independent stack evidence.	N/A — no production change.

Work Unit Evidence

Evidence	Result
Focused test	NODE_ENV=test npx --no-install tsx --test tests/ocr/e2e.test.ts — exit 0; 2 passed, 0 failed.
Runtime harness	The focused suite started bounded ephemeral localhost Express servers and exercised real HTTP requests through ingest, status, review, approval, and retrieval routes; all servers closed through test cleanup.
Complete gate	npm run check, npm run build, and NODE_ENV=test npm test exited 0; canonical Node passed 78/78. PYTHONDONTWRITEBYTECODE=1 ocr-service/.venv/bin/python -m pytest ocr-service/tests -q passed 13/13 offline with one dependency deprecation warning.
Rollback boundary	Remove tests/ocr/e2e.test.ts and revert only tasks 7.1–7.3 plus this Unit 13 progress/history metadata; preserve Units 1–12.

Validation and Boundary

• Unit 13 adds 183 functional lines. Required tasks/progress/history metadata adds 52 changed lines, for 235 total, below the 400-line budget.
• Tracked and complete intended-untracked whitespace checks passed. No matching tsx --test, E2E Node, or OCR Uvicorn process remained after the corrected self-excluding process check.
• Intended untracked inventory: src/modules/ocr/{dispatcher,indexing,retention,review}.ts; tests/ocr/{contracts-deploy,dispatcher,e2e,retention,review}.test.ts.
• Tasks 7.1–7.3 are complete. Task 7.4 remains unchecked and untouched; no production flag, service, database, vector store, embedding provider, OCR endpoint, commit, push, or PR was used.

Unit 14a: OCR Job Execution and Result

• Persisted accepted PDF bytes/results in SQLite, resumed queued jobs on idempotent resubmission, executed the existing renderer through an injected engine, recorded terminal status/errors, and added authenticated result retrieval; task 7.4 remains pending.

TDD Cycle Evidence

Task	Test File	Safety Net	RED	GREEN / TRIANGULATE	REFACTOR
Unit 14a	ocr-service/tests/test_api.py	13/13	8/10 missing execution/result; recovery triangulation 9/10 remained queued	10/10; success, restart recovery, unauthorized, missing, not-ready	Guarded duplicate workers; 10/10 remained green

Work Unit Evidence

Evidence	Result
Focused/regression	API 10/10; OCR 15/15; Node 78/78; npm run check and whitespace clean.
Runtime harness	Local Uvicorn with deterministic engine returned upload 202, status 200/succeeded, result 200, matching job/document identity; process/temp cleanup verified.
Rollback boundary	Revert ocr-service/app/main.py, ocr-service/tests/test_api.py, and this Unit 14a/history metadata; Units 1–13 remain intact.

Unit 14b: OCR Transient Cleanup

• Added authenticated idempotent client deletion after durable RAG result acceptance and automatic 24-hour OCR row/input/result expiry through the existing 30-second readiness healthcheck; cleanup failures fall back to TTL without failing durable candidates.

TDD Cycle Evidence

Task	Test Files	Safety Net	RED	GREEN / TRIANGULATE	REFACTOR
Unit 14b	ocr-service/tests/test_api.py; tests/ocr/{client,dispatcher}.test.ts	Python 10/10; Node 15/15	Python 10/11 lacked clock/sweep; Node 15/17 lacked delete client/dispatch	Python 11/11; Node 17/17; pre-persistence failure retained remote state and cleanup failure preserved local success	Reused healthcheck and generic retry/auth boundaries; focused suites remained green

Work Unit Evidence

Evidence	Result
Focused/regression	Python API 11/11; OCR 16/16; focused Node 17/17; canonical Node 80/80; check/build/whitespace clean.
Runtime harness	Local Uvicorn returned pre-expiry 200, post-24h 404, unauthorized delete 401, idempotent authenticated deletes 204/204, and post-delete 404; process/temp cleanup verified.
Rollback boundary	Revert only Unit 14b deltas in OCR API/client/dispatcher/tests and this progress/history metadata; durable RAG artifacts, active corpus, and Unit 14a remain intact.

Unit 14c: Logical Upload Source Reference

• Added an optional trimmed multipart sourceRef that remains independent from the temporary physical upload path, preserves original-filename fallback, rejects blank values, and documents the contract without activating content.

TDD Cycle Evidence

Task	Test Files	Safety Net	RED	GREEN / TRIANGULATE	REFACTOR
Unit 14c	tests/ocr/{upload-source-ref,contracts-deploy}.test.ts	Existing upload/contracts 13/13	2/4; missing contract and blank reference accepted	4/4; provided/trimmed reuse, physical-name independence, fallback, blank rejection	Shared one parsed value across file/ZIP branches; focused suite remained green

Work Unit Evidence

Evidence	Result
Focused/regression	Focused 4/4; multipart runtime 1/1; canonical Node 81/81; check/build/whitespace clean.
Runtime harness	Ephemeral localhost Express multipart uploads reused Errores Junio 2026 - OCR verificado.md across two physical PDF names, preserved fallback.pdf when omitted, rejected blank input 400, kept activation false, and removed temporary files.
Rollback boundary	Revert only Unit 14c deltas in src/{app,api/openapi}.ts, its two tests, and this progress/history metadata; Units 14a–14b, active content, and corpus remain intact.