On a case file that advice nearly works. Eleven PDFs about one insurance claim come to sixty-odd pages, which fits inside a 200,000-token window several times over.

It still misses the two questions the person handling the claim has. Is the second repair invoice in the folder at all, and does the date of loss on the claim form match the one in the adjuster’s report?

The reason is in the shape of the thing. A case file is a bundle of unlike documents about one entity, and the unit of work is the bundle. A claim, a credit application, a medical record, a hiring file. The contract, the certificate, the photographs, the expert’s report, the letters: nothing in there resembles anything else, and the answer comes from reading across all of it.

So neither question is a search. The first is about a document that does not exist, and no amount of context holds it. The second needs one value out of each of two files, compared. Models read the middle of a long input less reliably than the ends, and that has been measured. In a bundle of eleven, both values are in the middle.

This article works through what that changes:

what separates a case file from a collection that happens to be small
the list of pieces a case type expects, which indexes what should be there rather than what is
why a missing document is a first-class answer, and what it takes to produce one
the checks that compare one piece against another, and the typed values they need first
the shape of the answer, which is the state of the case rather than a passage
why this shape has the least prior art of the three, and how to evaluate it anyway

This article is part of Part IV of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. Part IV is where the input stops being a file and becomes a folder, and this is the folder that behaves like one object rather than like a collection.

🧭 New to the series? Start with the map: Prompt, Context, Loop sets out the three engineering layers every RAG system is built on, the prompt (the call itself), the context (what fills the model’s window), the loop (when the next call fires and when it stops), and walks the whole series through that lens, article by article. It is the shortest way to see what is covered and where this one sits.

📓 Run the completeness check on a folder of your own in the companion notebook: write down the pieces one case type expects, point the check at a folder, and print the three lists that come back: the pieces present, the pieces missing, and the files that belong to some other case. Repo → doc-intel/notebooks-vol1.

The worked example is a fire claim on a small joinery workshop, and it is fictional: the workshop, the dates, the amounts and the files were authored for the series and match no real insured or insurer. The list of expected pieces is written the way a broker’s back office would write one, not copied from anybody’s manual.

One folder first, then the comparison that says why the other two architectures do not fit it.

A fire damages a joinery workshop on 12 March. Over the next six weeks the broker’s folder for that claim fills up with eleven PDFs: the claim form, the policy schedule in force that month, a certificate of premium payment, the loss adjuster’s report, two repair quotes, one repair invoice, a set of damage photographs, two letters, and a vehicle schedule that belongs to a different policy and was filed here by mistake.

Nothing in that list resembles anything else in it. The photographs carry no text. The adjuster’s report is prose. The quotes are tables. The claim form is boxes with values typed into them.

Ask the handler what she does first and she does not search. She runs down a list she knows by heart and ticks off what is there. Fire claim, so the fire brigade report is required, and it is not in the folder. Two quotes accepted, so two invoices should follow, and only one has. That is a couple of minutes of work, and no document has been read yet.

Only after that does she read. The date of loss on the claim form says 12 March. The adjuster writes that he inspected the site on 3 March. One of those two is wrong, and until somebody says which, the file does not move.

Article 14A (three kinds of corpus, and what building for the wrong one costs) sets out three shapes a collection can take and a three-question test that tells you which one you are holding. This is the third shape: documents that arrive in bundles, several of them about one case.

Put the three side by side, question by question.

Two rows matter most here.

The index row. Article 14C (one document type, many copies, and the columns the business can already name) builds a table with one row per document and the business fields on it. That table describes what exists. Here the row is the case, and what goes in it is a list of pieces the case is supposed to hold, which is a different thing to write down and a different thing to fill.

The empty result. On the other two shapes, nothing found means the retrieval step failed and you go and fix it. On a case file, nothing found can be the correct answer, and often it is the answer the question was after.

The remaining comparison is the one that catches people out, because from a distance the two look identical. Article 14B (the folder whose documents share no field, indexed by one summary line each) treats a heterogeneous folder as one long document and routes down its nested outline. A case file is heterogeneous in exactly that way, so the routing runs fine here. It answers the wrong question: routing picks a piece, and these questions are about the set.

The preparation for this shape is one short artefact: a list of the pieces the case type demands, with how many of each and under what condition.

This is parsing, one floor up. On a single PDF, the parsing brick returns a small set of relational tables, one row per line, one row per section, one row per table cell (Article 5B, the relational tables RAG needs). A folder parses the same way: one table of expected roles, one table of arrived files with the role each was assigned and a confidence on that assignment. Most of the work on a case file is here, in structuring and cleaning the bundle into those two tables. The completeness check, the contradiction check and the case state later in this article are all queries over them.

A list of what should be there, rather than a description of what is. The index in Article 14C describes documents that exist; this list names documents that ought to, and the two are read at different moments.

An index built by reading the corpus describes what the corpus contains. Point an extraction pass at five thousand contracts and you get a table with one row per contract and a client, a date and a premium on each. Ask it which contract is missing and the question does not parse. Everything in the table is there by construction.

The list here is written before the folder is opened, and it comes from the process rather than from the files. A property damage claim needs a claim form, a policy schedule, proof that the premium was paid, an adjuster’s report, at least one quote, an invoice for each accepted quote, photographs, and a fire brigade report when the cause is fire. That list holds for the case type whether or not a single one of those documents was ever filed.

Derive the same list from the folders you already have and the missing piece disappears. Every role in such a list is a role that showed up in at least one folder, so a piece nobody ever files is simply not in the list.

Four fields per piece: what it is, how many are expected, when it is required, and whether the case can move without it.

```

Written from the process, before any folder is opened.

class ExpectedPiece(BaseModel):
"""One row of what a case type demands, filed or not."""
role: str # the business word: "loss adjuster report"
min_count: int # 0 when the piece is optional
max_count: int | None # None when any number of them is fine
required_when: str | None # a condition on the case's own fields
blocks_payment: bool # can the case move on while this is absent
PROPERTY_DAMAGE_CLAIM = [
ExpectedPiece(role="claim form", min_count=1, max_count=1,
required_when=None, blocks_payment=True),
ExpectedPiece(role="fire brigade report", min_count=1, max_count=1,
required_when="cause_of_loss == 'fire'",
blocks_payment=True),
ExpectedPiece(role="repair invoice", min_count=1, max_count=None,
required_when="accepted_quotes >= 1",
blocks_payment=True),
ExpectedPiece(role="correspondence", min_count=0, max_count=None,
required_when=None, blocks_payment=False),
]
``required_when` is what stops the list producing false alarms. The fire brigade report is required on a fire and irrelevant on water damage, so a flat list of required pieces complains on every claim that is not a fire. A check that is wrong most of the time gets switched off. The condition reads fields off the case itself, which means the case has to carry a handful of fields of its own before the completeness check can run at all.

blocks_payment is what makes the output usable. Two pieces are missing and one of them stops the money: a handler needs to see which, and a boolean per piece is enough to say it.

Where the list comes from is a conversation, and a short one. Somebody in the back office already has it, on a laminated card or in a procedures document, because a person has to check the same thing by hand today. Article 14C asks the business which fields it filters on. Here the question is narrower: what has to be in the folder before you sign this off. The answer usually comes back as a list.

With the list on one side and eleven files on the other, the completeness check is a join with three outcomes.

A piece is present when at least min_count files were assigned to its role. Seven of the nine roles pass here.

A piece is missing when the role is required, its condition holds, and no file was assigned to it. Two roles fail: the fire brigade report, because the cause is fire and nothing was filed, and the second repair invoice, because two quotes were accepted and one invoice arrived.

A file is unmatched when no role in the list fits it. Here that is the vehicle schedule from another policy. The outcome is worth reporting rather than dropping, because it is either a filing mistake or a role the list forgot.

The join rests on one step that can go wrong. Assigning a role to each file is a classification, and classifications make mistakes. An invoice filed under quotes reads as a missing invoice plus an extra quote, so the report then carries two wrong lines instead of none. The role assignment therefore keeps its own confidence, and a low-confidence assignment goes to the handler as a question rather than as a verdict.

Both of the questions the handler asked in section 1.1 have the same property: no passage anywhere in the bundle answers them, so no amount of ranking gets you there.

Ask a document-by-document pipeline where the fire brigade report is and it returns an empty result. Ask it the same thing on a claim where the report is filed under a strange name and it also returns an empty result. Those are different situations and the pipeline cannot tell them apart, because an empty result carries no information about why it is empty.

The completeness check can, and that is what the list is for. It knows the role exists, it knows the condition holds, and it knows nothing was assigned. The answer it produces is a sentence with a reason attached: the fire brigade report is required because the cause of loss is fire, no file in the folder was assigned to that role, and payment is blocked until one is.

Getting there needs three things the pipeline does not have by default. A list of roles, so absence has a name. A condition on each role, so absence is only claimed when the piece was actually due. And a confidence on the role assignment, so a badly filed document is reported as doubt rather than as a categorical no.

That last one matters more than it looks. A report that says missing when the document is sitting in the folder under a bad name is worse than no report, because the handler then asks the client for a file the client already sent. The series already has this discipline one level down, on the generation side. A typed answer contract carries not-found as a value rather than as an empty string, which is what the seven extraction patterns work out on a single document. Here the same move runs one floor up, on the bundle instead of on the field.

The second question compares. The date of loss appears on the claim form and again in the adjuster’s report, and the two do not agree.

Top-k retrieval is the wrong instrument here, and not because of its quality. Ranking returns the passages most like the question. A contradiction is not like anything: both halves read as ordinary on their own, neither passage mentions the other, and nothing in either one is closer to the question than the rest of the file.

What works instead is mechanical. Declare which fields are worth comparing and between which roles, extract that field from each piece with a typed contract and a citation, then compare the values.

Three details decide whether this works in practice.

The pairs are declared, not discovered. Nine pieces make thirty-six pairs, and multiplied by every field on every piece the comparison space is large and mostly pointless. What people actually check is a handful of pairs per case type, and a business user can name them in the same conversation that produced the list of expected pieces.

The values are typed before they are compared. 12/03/2025 and March 12, 2025 are the same date and two different strings. Alpine Joinery SARL and Alpine Woodworks are two different strings that may or may not be the same company. The comparison runs on parsed values, and the second case, where normalisation is a judgement rather than a format, is the one that needs a person in the loop. Turning a raw extraction into something joinable is its own piece of work, and Article 15 (preparing a corpus, one column at a time) takes it up.

Every value keeps its citation. A report that only says the dates disagree gives the handler nothing to go and check. One that says the claim form gives 12 March on page 1 and the report gives 3 March on page 4 can be settled in ten seconds.

Ask a corpus a question and a good answer names documents. Ask a case file and a good answer describes the case. That answer is a composed object rather than a passage.

Four lists and a verdict. The lists are what section 2 and section 3 produce; the verdict is the sentence a handler would write at the top of the file.

class FieldConflict(BaseModel): """One declared pair that came back with two values.""" field: str # "date_of_loss" left: TypedValue # value, doc_id, page, quoted line right: TypedValue class CaseState(BaseModel): """What a question about a case returns: the whole bundle.""" case_id: str case_type: str # "property damage claim" present: list[MatchedPiece] # role, doc_id, match confidence missing: list[MissingPiece] # role, why it was due, what it blocks unmatched: list[str] # files no role in the list fits conflicts: list[FieldConflict] # the declared pairs that disagree verdict: Literal["complete", "incomplete", "conflicting"] verdict_reason: str # one sentence naming what blocks
Two properties of that object matter for how it gets filled.

It is a list-shaped answer, not a top-ranked one. Every entry counts and the ordering does not, which is the same contract a listing question needs inside one document. Loop engineering for listing questions works that out on a single PDF, and the constraint carries over to a bundle unchanged.

It is filled piece by piece rather than in one call. Eleven documents do fit in a window, and putting them all there is still the worse option: the extraction is per piece, the citation has to name the piece it came from, and reading one document at a time is what keeps both of those true. That trade, one call over everything against one call per item, is the subject of iterating top-k one at a time on a single document.

The object needs one more thing, and the series already has it. A piece can point at another piece: the adjuster’s report says see the accepted quote, and the value being compared is in the other file. That is the same fetch-and-continue loop as a document that answers ‘see Section 7.2’, run across files instead of across sections.

Of the three shapes, this is the one with the least prior art, both in the series and in the retrieval literature. So the failures below come with fewer known fixes than the ones in the other two articles.

The last row is the important one. Retrieval benchmarks measure whether the right passage came back. There is no standard set that measures whether a system correctly said a required document was never filed. The closest thing in the literature is the unanswerable half of a reading comprehension set, which is a different question asked of a single passage. So the evaluation stays local: take fifty cases a handler already checked by hand, run the completeness check, and count the two error types separately. A piece wrongly called missing costs a phone call to the client. A missing piece not caught means the claim is settled on an incomplete file, which is the thing the check exists to prevent.

Three pieces of the build are not in this article. The index that holds the case and its fields is Article 15 (preparing a corpus, one column at a time). The vocabulary that says a loss adjuster report and an expert report are the same role is Article 16 (where the corpus vocabulary comes from). Routing a user’s question to the right case, and to the right mode once it is there, is Article 17 (querying a corpus, filter first and retrieve second).

A case file is neither a document nor a corpus. It is a bundle of unlike pieces about one entity, and the unit of work is the bundle.

The move that carries over from the single-document articles is relational. Parse the bundle into tables first, and every later question becomes a query; the model calls come after the structure, not instead of it.

Two things follow, and both are cheap. The index is a list of what the case type demands, written from the process before any folder is opened, which is what gives an absent document a name. And the answer is the state of the case, four lists and a verdict, each entry carrying the file and the page behind it.

The two questions worth building for are not retrieval questions, and that is the awkward part. A missing document is not in the corpus by definition, and a contradiction is a comparison between two typed values rather than a passage to rank. Both are easy to compute once the list exists and the pairs are declared, and neither happens unless you decide to build it.

If you have a folder like this, the first experiment costs an afternoon. Write down the pieces one case type expects, take twenty real cases, and run the join by hand. The count of missing pieces is the size of the problem, and it is worth knowing before anyone writes ingestion code.

Article 14A (three kinds of corpus, and what building for the wrong one costs) has the test that tells you whether this is the shape you are holding. Article 14B (the folder whose documents share no field) and Article 14C (one document type, many copies) take the other two.

Earlier in the series:

Document Intelligence: series intro. What the series builds, brick by brick, and in what order.

What works, what breaks

Baseline Enterprise RAG, from PDF to highlighted answer. The four-brick pipeline end to end: PDF in, highlighted answer out.
Embeddings Aren’t Magic: The Predictable Failure Modes of RAG Retrieval. Where embedding similarity wins (synonyms, typos, paraphrase), where it predictably breaks (unknown terms, negation, term-vs-answer relevance), and how to use it anyway.
RAG is not machine learning, and the ML toolkit solves the wrong problem. Why chunk-size sweeps and finetuning optimize the wrong thing; route by question type instead.
From regex to vision models: which RAG technique fits which problem. Two axes, document complexity and question control, that pick the technique for each case.
10 common RAG mistakes we keep seeing in production. Ten production mistakes, organized brick by brick, with the fix for each.

Document parsing

Building Document Structure with Loop Engineering: Recovering a PDF’s Outline from Body Typography for RAG. Rebuilding the outline from body typography when the PDF ships no contents page at all: six signals, one bounded loop.
Before Full Agentic RAG: Know How You Decide, and the Parsing Methods You Pick From. The parsing methods as a catalogue, and the decision of which to run, before handing the loop to an agent.

Question parsing

Loop engineering for RAG question parsing: the small loop that runs before retrieval. The bounded loop that runs before retrieval: parse, check the fields make sense, re-ask once when they do not.

Generation

Loop Engineering for RAG Generation: Iterate top-k One at a Time. Reading the retrieved pages one at a time instead of all at once, and what that buys when the answer sits in only one of them.
Most RAG Hallucinations Are Extraction Errors: Seven Patterns for a Typed Generation Contract. Seven recurring ways a model gets the extraction wrong, and the typed contract that catches each one.
Loop engineering for RAG generation: an LLM cascade from a cheap local model up to a hosted flagship. Starting on a cheap local model and escalating only when the answer does not hold up, measured.

One-document pipelines

Prompt Engineering Isn’t Enough: How Four Bricks of Context Engineering Stop RAG Hallucinations. Why a better prompt does not fix a wrong page, and what each of the four bricks contributes to the context instead.
Cut an Enterprise RAG Pipeline’s Latency and Cost by Calling the LLM Less, Not by Buying a Faster Model. Cutting a pipeline’s latency and cost by calling the model less often and cheaper, not by buying a faster one.
Loop engineering for cross-references: when RAG answers ‘see Section 7.2’ instead of the actual answer. When the answer says “see Section X”, the pipeline loops back and fetches it.
RAG workflow and loop engineering: the dispatcher that decides when to loop and when to stop. Feedback loops, bounded iteration, and the dispatcher, composed into one workflow.
Loop engineering for RAG: the small loops inside each step, the big loops across the pipeline. The two scales of loop: small bounded loops inside each brick, big generation-triggered loops across them.

Also referenced above, listed here without links: Article 14A (three kinds of corpus, and what building for the wrong one costs), Article 14B (the folder whose documents share no field), Article 14C (one document type, many copies), Article 15 (preparing a corpus, one column at a time), Article 16 (where the corpus vocabulary comes from), and Article 17 (querying a corpus, filter first and retrieve second).

External sources:

Scott Barnett et al., Seven Failure Points When Engineering a Retrieval Augmented Generation System, 2024 (arXiv 2401.05856). Missing content is their first failure point, measured on three production systems. On a case file it stops being a failure and becomes the output.
Pranav Rajpurkar, Robin Jia and Percy Liang, Know What You Don’t Know: Unanswerable Questions for SQuAD, 2018 (arXiv 1806.03822). Fifty thousand questions written so that the right answer is to abstain. The nearest thing in the benchmark literature to scoring a system on absence, and it is still about one passage rather than about a bundle.
Harsh Trivedi et al., MuSiQue: Multihop Questions via Single-hop Question Composition, 2022 (arXiv 2108.00573). Questions built so that no single passage can answer them, which is the property every cross-piece check in section 3.2 has.
Yuta Koreeda and Christopher D. Manning, ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts, 2021 (arXiv 2110.01799). 607 annotated contracts labelled entailed, contradicting or not mentioned, with the evidence spans. The document-level version of the contradiction check, and useful for how hard they found it.
Nelson F. Liu et al., Lost in the Middle: How Language Models Use Long Contexts, 2023 (arXiv 2307.03172). Performance is highest when the relevant text sits at the start or the end of the input and drops in between, which is the measured reason the whole bundle in one prompt is not the fix.

Newspaper

Parse the Folder, Not Just the PDFs: The Relational Tables RAG Needs on a Case File

Written from the process, before any folder is opened.