Blueprints BP-03
Document intelligence pipeline
Facts are locked in PDFs, scans and email attachments.
01 / Fit
Use it when
- The same kinds of document arrive in volume, in many layouts.
- People currently re-key fields from documents into another system.
- Each field has a business rule or a cross-check that can confirm it.
- Downstream systems need rows and lineage, not summaries.
Avoid it when
- The source can send structured data instead. Ask for the feed before parsing the PDF.
- There are only a few layouts and they never change. Templates are cheaper and exact.
- Nobody is available to review flagged documents. Without review the errors accumulate unseen.
02 / The diagram
Run it. Then run the failure path.
Select a node to see what it does, how it fails and what we measure.
- Ingest (Files in). Collects files from inboxes, portals and shared drives, and fingerprints each one. Fails when: The same document arrives twice, or a new source format appears. Safeguard: Content-hash de-duplication. Unknown formats are quarantined, not dropped.
- Classify (Document type). Decides what the document is, so the right schema applies. Fails when: A new layout looks like an old type. Safeguard: A confidence threshold. Uncertain documents go to review with the top two guesses.
- Parse (Layout and OCR). Recovers reading order, tables and key-value pairs from layout and OCR. Fails when: Scans are skewed, stamped or handwritten. Safeguard: A second parser for low-quality pages, and a quality score per page.
- Extract (Typed fields). Fills a typed schema from the parsed document, with a source span for every field. Fails when: The model returns a plausible value that is not in the document. Safeguard: Every field must cite its span. A field without one is left empty.
- Validate (Business rules). Checks extracted fields against business rules and against each other. Fails when: Totals do not add up, or a date is out of range. Safeguard: A failed check sends the document to review. Nothing partial is stored.
- Store (Rows, lineage). Writes validated rows to the warehouse with lineage back to the source page. Fails when: A schema change breaks a downstream consumer. Safeguard: Versioned schemas and contract tests.
- Review queue (Correct fields). A person corrects flagged fields with the source page beside them. Fails when: The queue grows faster than it is cleared. Safeguard: Alert on queue age. Corrections feed the golden set.
- Golden set (Checked cases). Holds reviewed documents with known-correct fields, replayed before each release. Fails when: It stops reflecting the documents that arrive today. Safeguard: Refreshed from reviewed cases on a schedule.
Read the diagram as text
- Ingest (Files in). Collects files from inboxes, portals and shared drives, and fingerprints each one. Fails when: The same document arrives twice, or a new source format appears. Safeguard: Content-hash de-duplication. Unknown formats are quarantined, not dropped.
- Classify (Document type). Decides what the document is, so the right schema applies. Fails when: A new layout looks like an old type. Safeguard: A confidence threshold. Uncertain documents go to review with the top two guesses.
- Parse (Layout and OCR). Recovers reading order, tables and key-value pairs from layout and OCR. Fails when: Scans are skewed, stamped or handwritten. Safeguard: A second parser for low-quality pages, and a quality score per page.
- Extract (Typed fields). Fills a typed schema from the parsed document, with a source span for every field. Fails when: The model returns a plausible value that is not in the document. Safeguard: Every field must cite its span. A field without one is left empty.
- Validate (Business rules). Checks extracted fields against business rules and against each other. Fails when: Totals do not add up, or a date is out of range. Safeguard: A failed check sends the document to review. Nothing partial is stored.
- Store (Rows, lineage). Writes validated rows to the warehouse with lineage back to the source page. Fails when: A schema change breaks a downstream consumer. Safeguard: Versioned schemas and contract tests.
- Review queue (Correct fields). A person corrects flagged fields with the source page beside them. Fails when: The queue grows faster than it is cleared. Safeguard: Alert on queue age. Corrections feed the golden set.
- Golden set (Checked cases). Holds reviewed documents with known-correct fields, replayed before each release. Fails when: It stops reflecting the documents that arrive today. Safeguard: Refreshed from reviewed cases on a schedule.
03 / Components
Every component, and how it fails.
| Component | Job | Typical choices | Fails when | Safeguard |
|---|---|---|---|---|
| Ingest | Collects files from inboxes, portals and shared drives, and fingerprints each one. | Connectors for mailboxes, portals and shared drives, with content hashing | The same document arrives twice, or a new source format appears. | Content-hash de-duplication. Unknown formats are quarantined, not dropped. |
| Classify | Decides what the document is, so the right schema applies. | Document classifier with a confidence threshold | A new layout looks like an old type. | A confidence threshold. Uncertain documents go to review with the top two guesses. |
| Parse | Recovers reading order, tables and key-value pairs from layout and OCR. | Layout-aware parser and OCR engine, with a second parser for poor scans | Scans are skewed, stamped or handwritten. | A second parser for low-quality pages, and a quality score per page. |
| Extract | Fills a typed schema from the parsed document, with a source span for every field. | Language model with schema-constrained output and source spans | The model returns a plausible value that is not in the document. | Every field must cite its span. A field without one is left empty. |
| Validate | Checks extracted fields against business rules and against each other. | Rule engine for field checks and cross-field checks | Totals do not add up, or a date is out of range. | A failed check sends the document to review. Nothing partial is stored. |
| Store | Writes validated rows to the warehouse with lineage back to the source page. | Data warehouse with versioned schemas and lineage columns | A schema change breaks a downstream consumer. | Versioned schemas and contract tests. |
| Review queue | A person corrects flagged fields with the source page beside them. | Review interface with the source page beside the extracted fields | The queue grows faster than it is cleared. | Alert on queue age. Corrections feed the golden set. |
| Golden set | Holds reviewed documents with known-correct fields, replayed before each release. | Versioned set of reviewed documents and a replay harness | It stops reflecting the documents that arrive today. | Refreshed from reviewed cases on a schedule. |
04 / Budgets and evals
What the design is tuned for.
Field-level accuracy, cost per document. These are design targets, not results.
| Stage | Design target |
|---|---|
| Parse | The low-cost parser handles clean pages. The heavier one is kept for pages below the quality score. |
| Extract | Cost per document. One schema-constrained call per document type where possible. |
| Validate | Rule-based, so every failure can be explained and repeated. |
| Review | Time per document. Only flagged fields are shown, with the source beside them. |
Eval plan 01
Golden set
Reviewed documents with known-correct fields, spread across document types, layouts and scan quality. It includes handwritten, stamped and multi-page table cases.
Eval plan 02
Release gates
Field-level accuracy per field, not one average. Share of fields with a valid source span, and review rate at the current thresholds.
Eval plan 03
Catching regressions
The golden set is replayed before each release and compared field by field with the last one. Corrections from the review queue are added on a schedule.
05 / Design notes
Where the whiteboard and production differ.
- Document-level accuracy hides the problem. One field in a new layout can fail every time while the average stays high.
- Tables that span pages, and totals that depend on them, cause more errors than free text.
- A new layout usually arrives without notice and is classified as an old type. Confidence thresholds catch this only if they are tuned per type.
- Scan quality varies more by source than by document type. A quality score per page is worth storing.
- Review effort is the cost that decides whether the pipeline pays for itself, so the review screen needs as much design as the model prompt.
06 / Deploy
Deploy this with us.
A pod can take BP-03 from this page to your stack. Tell us the workflow and we will tell you what changes for your systems.
You will talk to an engineer