Prompt, Context, Loop: The Three Engineering Layers Every RAG System Is Built On

Prompt engineering is the call itself: the system message, the instructions, the schema that fixes the output shape. Context engineering is what fills the model’s window: retrieval, compression, deciding what to keep out. Loop engineering is what happens around the call: when the next one fires, when the loop stops, how the system recovers when a check fails. Almost every argument about RAG is really an argument about which of these three layers you are standing on. Name the layer and most of the confusion clears. This article is the map: what each layer owns, how they map onto the series, and why the tidy “one replaced the next” story is only half true.

This article is a manifesto of Enterprise Document Intelligence, a series that builds an enterprise RAG system from four bricks. It treats the three-layer framing (prompt engineering, context engineering, loop engineering) that has become the dominant 2026 narrative, and asks the harder question: is the evolution from one layer to the next actually a sequence, or is it a retrospective story imposed on patterns that were always there?

📓 The series’ companion notebooks live on GitHub at doc-intel/notebooks-vol1. Each one runs a brick end to end on a real PDF, so you can watch these three layers play out: the prompt that fixes the answer shape, the context assembled from a parsed question and retrieved pages, the loop that retries when a check fails.

Three disciplines stack on top of an LLM call. Each one is responsible for a different lever; together they describe most of what production teams do when they go past hello-world.

Prompt engineering is the layer everyone meets first. The model needs a system message that sets its role and constraints, a user message that carries the question and the context block, optionally a schema that fixes the shape of the expected output. Writing those three pieces well is the difference between a model that follows the rules and a model that improvises. The discipline shipped in 2022-2023 with GPT-3.5 and ChatGPT; the term prompt engineering installed in the public discourse around the same time.

Context engineering is the layer that ships when the prompt alone is no longer enough. The model has a finite context window; the practitioner decides what fills it. Retrieval picks the relevant documents. Compression removes the noise. Isolation keeps sub-agent outputs out of the main window. The four canonical strategies (LangChain’s write, select, compress, isolate) name what the practitioner has been doing implicitly since the first RAG paper. The term installs in 2025 (Karpathy and Tobi Lütke use it publicly, Anthropic publishes the canonical Effective context engineering for AI agents in 2025).

Loop engineering is the layer that ships when a single call is no longer enough. The model produced a plausible answer that fails the schema. The listing returned twelve items but the model itself flagged the answer as incomplete. The API timed out. The agent picked the wrong tool. The practitioner now owns four control surfaces: what triggers the next call, when the loop stops, how the system recovers when a call fails, how independent agents verify the result before committing. The name installs in May 2026 with Boris Cherny’s “I don’t prompt Claude anymore. I have loops running that prompt Claude” and Anthropic’s launch of Dynamic Workflows ten days later.

The narrative writes itself: prompt engineering became context engineering became loop engineering, each layer added once the previous one was saturated. It is a clean story. It is also slightly wrong.

The patterns of each layer existed before the name. ReAct (Princeton and Google, October 2022) is the canonical reasoning-plus-action loop, more than three years older than the term loop engineering. AutoGPT made autonomous goal-driven loops public in March 2023. Reflexion added self-evaluation at NeurIPS 2023. Geoffrey Huntley’s Ralph Loop put goals on disk in July 2025. By the time loop engineering installed as a term in May 2026, the patterns had been in production for more than three years.

The context-engineering side is similar. The original RAG paper (Lewis et al., 2020) is two years before prompt engineering even installed as a term. By 2022 every serious LLM application had a context-management story (chunking, retrieval, deduplication, truncation) that no one called context engineering yet. The LangChain four-strategy taxonomy (write, select, compress, isolate) names what was happening implicitly in every production RAG system for four years.

The honest reading is that all three layers existed simultaneously from the start of the LLM era. What changed across the four-year window is which layer was the dominant bottleneck in production. As long as the bottleneck was prompt quality, no one paid attention to the loop. As soon as the bottleneck shifted to context discipline, the LangChain taxonomy crystallised. As soon as the bottleneck shifted again to agent reliability over many turns, the Anthropic dynamic-workflows catalogue followed.

The evolution-of-disciplines framing makes for a tidy slide deck. The bottleneck-moves framing is what actually describes what changed.

Three forces push the bottleneck up the stack over time.

Models get better at their layer. Each generation of models requires less prompt engineering to produce a fluent answer. GPT-4 followed instructions that GPT-3 ignored. Claude 3.5 returned valid JSON that earlier models would have wrapped in prose. The prompt-engineering layer is not gone; it just stopped being the rate-limiter. The next generation of models will retire most of what counts as context engineering today (longer effective context windows, less mid-context attention degradation, native multi-document reasoning). The layer above is what is left.

Window sizes grow. Early-2023 models shipped 4k context windows; 100k arrived within the year (Claude 2), a million-token window by 2024 (Gemini 1.5), and a million is a headline number by 2026. The headlines still oversell effective recall (a model with a 1M window suffers from context rot well before the nominal limit), but the engineering pressure on context management changes shape. The choice is no longer how to fit it all in but what to keep out so the model can still find it. The bottleneck moves from packing to curation, which is exactly what select and isolate are about.

Production runs longer. A 2022 use case was a single-shot question answering session. A 2026 use case is an agent running for forty turns over six hours, spawning sub-agents that fan out and synthesise. The 2022 use case did not need loop engineering because there was no loop to engineer. The 2026 use case cannot survive without it. The bottleneck does not move because the practice changes; the bottleneck moves because the use case changes.

These three forces work together. Better models reduce the pressure on the layers below; longer windows shift the pressure to curation rather than packing; longer runs add a layer that did not exist before. The result is the sequence of names. It is less an evolution of techniques than a sequence of what hurts most in production.

The three-layer schema is most useful when it makes the boundaries clear. Each layer owns one question.

Layer 1 – prompt engineering owns what does the model read on this single call. The system message, the user message, the schema, the tool definitions, the conversation history bundled into the next call: everything between the opening {"role": "system"} and the response. The questions a prompt engineer answers are what role should I assign, what constraints make the output reliable, what schema fixes the output shape. The discipline is mature and bounded.

Layer 2 – context engineering owns what enters and exits the model’s context window between calls. Lance Martin’s LangChain taxonomy breaks it into four strategies. Write is the cached prefix that does not vary turn to turn (system prompt, tool definitions). Select is retrieval and memory recall: picking what is relevant now from a larger pool. Compress is summarisation, /compact, and per-tool truncation: shrinking what would otherwise overflow. Isolate is sub-agent calls in their own windows, memory tools that hold state outside the main context, programmatic tool calling where intermediate results never enter the context. The four strategies compose; the four together describe what context engineering does in production.

Layer 3 – loop engineering owns when the next call fires, when the loop stops, how the system recovers, and how the result is verified before it ships. The four control surfaces are: the trigger predicate (what condition fires the next call), the termination condition (loop-until-done versus loop-until-budget versus loop-until-dry), the recovery path (retry-with-backoff, fallback to a larger model, escalate to a human, skip the item and return what was computed), and adversarial verification (other agents try to refute the answer before it commits).

The boundaries between layers matter because each layer has its own tooling and its own failure modes. A bad prompt fails noisily: the model says something off. Bad context engineering fails quietly: the model says something fluent but wrong because the retrieved context was wrong. Bad loop engineering fails expensively: the loop spins on the same payload, burns the token budget, eventually times out. The three classes of failure need three different debugging practices; treating them as a single black box makes none of them tractable.

A note on harness engineering. Some practitioners split out a fourth term, harness engineering, for the execution environment between context and loop: the tools the model can call, the guardrails, the verification, the memory across sessions. In this three-layer framing those concerns are not missing, they are distributed. The tools and the memory sit in context engineering’s isolate strategy; the verification and the recovery sit in loop engineering. The companion manifesto M6 (harness engineering for RAG) takes the other cut, naming the harness outright and reading each of the four bricks as a small harness of its own (a set of methods plus the code that selects and validates). Three layers or four is a choice of zoom, not a disagreement about the system.

The 2026 loop-engineering catalogue settled on seven named primitives plus one structural rule. Anthropic’s Dynamic Workflows launch (May 2026) named six of them; retry-with-backoff is older and general. The rule is older and lives in the practitioner literature.

The patterns are not a sequence to walk through in order. They are a vocabulary. A real loop uses two or three of them composed. Fan-out a question across three sub-agents. Adversarially verify the consensus answer by asking a refuter agent to break it. Loop-until-done on a completeness predicate. Retry-with-backoff on transient failures. The composition is the engineering; the names are how a team talks about the engineering.

The rule that holds the catalogue together is simpler and predates the names. Every retry should change something. A loop that retries the same payload after the same failure is not learning. It is spinning. The change can be in the payload (the dispatcher widened the retrieval scope), the model (the small model failed, the large one gets the second shot), or the strategy (keyword retrieval missed, the second pass uses dense retrieval). A retry without one of these changes is the canonical waste of the loop-engineering layer.

The rule and the catalogue together describe most of what a production loop does. The discipline is not in the patterns alone; it is in which two or three patterns are composed for the case at hand, and in checking that each retry changes something.

The three-layer framing predicts its own obsolescence. If the bottleneck moves up the stack as the lower layers stabilise, the layer above loop engineering is what will be named next.

The candidates already have informal names but no consensus. Skill engineering (Anthropic’s late-2025 Agent Skills launch) treats the catalogue of capabilities an agent can deploy at runtime. Memory engineering treats the persistent state that survives across sessions (project conventions, prior decisions, user preferences). Goal engineering treats the long-horizon target an agent maintains across many loop iterations (Anthropic’s /goal command and OpenAI’s Codex CLI equivalent in May 2026). Tool catalogue engineering treats the meta-layer where the agent picks which tools to make available to itself for the next stretch of work.

It is not yet clear which of these will end up as the dominant fourth-layer name. The pattern from the previous three layers suggests the answer is whichever one becomes the production bottleneck first. The three forces from Section 3 will push it the same way they pushed the previous three.

A useful posture for a team reading this in 2026 is to stop investing too heavily in the names and instead get good at recognising when their bottleneck moves. If most production failures are the model says something off the team’s bottleneck is prompt engineering. If most failures are the model says something fluent but wrong the bottleneck is context engineering. If most failures are the loop tries the same thing four times the bottleneck is loop engineering. If most failures are the agent forgets what we agreed yesterday the bottleneck has moved past loop and into whatever-comes-next.

This is the map, and it is complete: every article in the series is below, in reading order, tagged with the layer it works on. You do not need a separate announcement post to see the whole plan. The layers do not arrive in a strict order as you build (Part II does context work across the input bricks before prompt engineering takes over at generation); they are a lens laid over the reading order. Published articles are linked; the rest are on the way.

1 – the minimal RAG: a PDF-in, highlighted-answer-out pipeline in about a hundred lines, no vector database and no framework. The whole four-brick loop in miniature.
2 – embeddings aren’t magic: the predictable failure modes of embedding retrieval, and why topical proximity is not the question-to-answer relationship.
2bis – rerankers aren’t magic either: when a cross-encoder reranker earns its extra latency, and when the dictionary already wins.
3 – RAG is not machine learning: why the ML toolkit solves the wrong problem for a retrieval-and-generation system.
4 – which technique fits which problem: from regex to vision models, a grid matching each RAG technique to the problem it actually fits.
4bis – ten common RAG mistakes: the production failure modes the rest of the series designs against.

Document parsing, choosing what to read out of the PDF (context):

5A – the two layers of a PDF: the text layer and the structure layer, and why extract_textthrows half of what drives RAG quality away.
5B – the relational data model: the relational tables (lines, pages, blocks) a PDF should become instead of flat text.

Beyond the default PyMuPDF parse, the brick reaches for heavier methods when a page needs them, each producing the same output tables:

5bis – Azure Layout: reaching for Azure Layout when PyMuPDF cannot see the table.
5ter – Docling: local parsing with Docling for rich tables, no cloud upload.
5quater – vision LLMs as parsers: using a vision model to read the charts and diagrams a text parser skips.
5quinquies – EasyOCR: OCR for scanned PDFs, and why free OCR gives you words, not a document.
5sexies – searchable images: making a PDF’s images searchable without paying to read them all.
5septies – reconstructing the TOC: rebuilding the table of contents a PDF forgot to ship, so retrieval can scope by section.

Then two loop-layer follow-ups, where parsing stops being one fixed pass and becomes adaptive:

5octies – the TOC as a loop: the TOC reconstruction reframed as a loop that reads the document top-down.
5nonies – agentic parsing: letting the pipeline choose its own parser per page.

Question parsing, structuring the question before you search (context):

6A – parse the question first: the missing step in most RAG pipelines, structure the question before retrieval touches it.
6B – the five fields to extract from a question: keywords, scope, shape, decomposition, and clarification.
6C – dispatch the parsed question: turning one parsed question into four routing decisions (chunk strategy, model tier, fragments, audit).
6bis – clarify vague questions: clarify a vague question once, then learn the default so you stop asking.
6ter – the untaught lessons of question parsing: the positions on the brick the tutorials skip.

Then the two companions that name the layers for this brick:

6quater – context engineering for question parsing: typing the raw question into fields that steer retrieval and generation.
6quinquies – the question-parsing loop: the small loop that runs before retrieval fires.

Retrieval, filtering rather than searching (context):

7A – retrieval is filtering: the mental model, retrieval as narrowing a scope, not searching an index.
7B – anchor detection: keyword, embedding, and TOC signals run in parallel, then one LLM call at the end.
7C – the LLM arbiter: an LLM picking the right candidate page, with reasons.
7bis – context engineering: the four typed inputs: the context layer named outright, the four typed inputs behind every answer.
7ter – the untaught lessons of retrieval: why cosine similarity is not the foundation it is treated as.
7quater – hierarchical retrieval: loop engineering to read a long document by its table of contents (loop layer).
7quinquies – retrieval stops hallucination: how the retrieval brick decides what the model is even able to invent.
7sexies – table-row retrieval: retrieving the right row from a wide table.

Generation, where prompt engineering takes over:

8A – the typed answer contract: the schema that prevents hallucination by refusing to return free text.
8B – prompt assembly: assembling each generation prompt from a base prompt plus the rules each question needs.
8C – validating the answer: checking spans and quotes, with the feedback loop, before the user sees the answer.

Then the loop-layer follow-ups, once one generation pass is no longer enough:

8bis – top-1 versus top-K generation: when a single generation pass is enough and when you need several (loop layer).
8ter – generation patterns: the recurring patterns for a typed generation contract.
8quater – the model cascade: an LLM cascade from a cheap local model up to a hosted flagship (loop layer).
9A – the production pipeline: the four upgraded bricks assembled into one pipeline, relational parsing to TOC retrieval to typed answers.
9B – one pipeline, four PDFs: the same pipeline run end to end on four very different documents.
9bis – when retrieval returns the wrong page: catching and recovering from a retrieval miss before it reaches the answer.
9ter – routing to a cheap model: sending the easy questions to a cheaper model to cut cost.
10A – the escalation cascade: adaptive parsing, start cheap and pay for a heavier parser only when the page needs it.
10B – escalation in action: the escalations walked end to end, a flat table to Azure, a figure to a vision LLM.
11 – cross-references: answering “see Section 7.2” with the section itself, not the pointer.
12 – listing questions: when the answer is every matching passage, not just the top one.
13 – the workflow dispatcher: the dispatcher that decides when to loop and when to stop.
13bis – loop engineering for the pipeline: the loop layer named explicitly for the composite dispatcher pipeline.

The four bricks lifted from one document to many. At corpus scope the work is still mostly context, choosing which documents and sections enter the window before the single-document pipeline runs:

14 – the corpus problem (context): Why Naive RAG Fails on Real Archives.What changes when you go from one document to a whole corpus.
15 – preparing the corpus (context): From a Folder of PDFs to a Queryable Corpus.Indexing, typing, and versioning up front.
16 – the corpus ontology (context): Why Your Enterprise RAG Needs an Ontology, Not a Knowledge Graph.The typed tags and relationships that make a corpus queryable.
17 – querying the corpus (context): Query the Corpus: SQL Filter First, Retrieval Second.Asking a question across many documents at once.
17bis – the corpus clarification loop (loop): Clarifying a Question at Corpus Scope.The single-document clarification loop lifted to the corpus.
17ter – context engineering for a corpus (context): Context Engineering for a Corpus of Documents.The context layer lifted to corpus scope.
17quater – loop engineering for a corpus (loop): Loop Engineering for a Corpus.The loop layer lifted to corpus scope.

Not a fourth layer but the operational cross-section: these run, measure, and secure the three layers rather than design one, so each one touches all three:

18 – code architecture: how the pipeline is structured in code, brick by brick.
19 – storage: long-format tables and replayable artefacts, with 19bis(schema migrations) and19ter(the storage map).
20 – evaluation: evaluation sliced by failure mode rather than one aggregate score.
21 – cost and latency: keeping both in check as the pipeline grows.
22 – security: access control and data protection around the corpus.
B01 – spelling correction: cleaning OCR and typo noise before retrieval.
B02 – FAQ as RAG: treating an existing FAQ as a retrieval corpus.
B03 – the justified “I don’t know”: refusing to answer, with the evidence for the refusal.
B04 – tables in PDFs: the deep dive on table parsing.
B05 – choosing models: which model to reach for at each brick.
B06 – the dispatched architecture: the dispatcher pattern in full.
B07 – faithful mocks: testing the pipeline with mocks that behave like the real calls.
B08 – a CV-parser benchmark: benchmarking parsers on real CVs.
B10-B12 – running locally: local LLMs with Ollama, local embeddings, and a local-model benchmark.
M1 – amplify the expert: the philosophy under the whole series, the system scales the expert’s judgment, it does not replace it.
M3 – the untaught lessons, article by article: every original position in the series, one section per article.
M4 – ten positions: the ten places the series breaks with mainstream RAG tutorials.
M5 – out-remembers and out-counts: machine learning does not out-think the expert, it out-remembers and out-counts them.
M6 – harness engineering for RAG: the same system read from the other side, the four bricks as a harness (a set of methods plus selection and validation), the model as one call inside the scaffolding.

The vocabulary helps team communication; do not let it become a checklist. A team that says “we need to add adversarial verification here” is having a more productive conversation than a team that says “the answers feel iffy sometimes”. The names earn their weight when they sharpen a discussion. They lose their weight when they become tick-boxes. The teams that produce the worst loop engineering are the ones that mechanically apply fan-out, tournament, and adversarial verify to every problem they meet. The good teams pick the two or three patterns that fit this case and ignore the rest.

Prompt, context, loop: three engineering layers on one LLM call, and the lens this whole series is built on. Section 7 is the map; every article that follows sits on one of the three, and knowing which one tells you what you are actually working on. The one caveat is the tidy evolution story. All three layers existed from the start of the LLM era; the names only landed in sequence, three years apart, as each layer in turn became the production bottleneck. Better models retired most of what was hard about prompt engineering; longer windows changed the shape of context engineering; longer runs created the loop-engineering layer that did not exist before. So the framing is two things at once: a durable way to organize the work, and a snapshot of where the bottleneck sits in 2026. The fourth layer will get named when its bottleneck arrives.

The practical posture for a team is to read the three-layer framing as a triage guide for what to debug when. If the model says something off, look at prompts. If the model says something fluent but wrong, look at context. If the loop spins, look at termination. If the agent forgets across sessions, the bottleneck has moved past loop. The names help. The honest reading of where the bottleneck is on this particular Tuesday afternoon helps more.

The three layers and their names.

Effective context engineering for AI agents, Anthropic Engineering. The canonical 2025 post that codified the term.
Context engineering for agents, Lance Martin, LangChain blog. The- write/- select/- compress/- isolatetaxonomy used as the spine of layer 2.
Anthropic’s Dynamic Workflows in Claude Code (May 28 2026). The launch that named the six loop patterns.
The Anthropic leader who built Claude Code says he ditched prompting: now he just writes loops, The New Stack. Boris Cherny’s verbatim quote that put- loop engineeringin front of a developer audience.

The lineage.

Agentic Loops: From ReAct to Loop Engineering (2026 Guide), Data Science Dojo. The historical walkthrough of ReAct (October 2022) → AutoGPT (March 2023) → Reflexion (NeurIPS 2023) → Plan-and-Execute → OODA → Ralph Loop (Huntley, July 2025) →- /goal(Claude Code, May 2026) → Dynamic Workflows (May 28 2026).
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Lewis et al., NeurIPS 2020 (arXiv:2005.11401). The original RAG paper. Predates- context engineeringthe name by five years.
ReAct: Synergizing Reasoning and Acting in Language Models, Yao et al., October 2022 (arXiv:2210.03629). The canonical reasoning-plus-action loop. Predates- loop engineeringthe name by more than three years.

Practitioner takes.

The Anatomy of an Agent Loop, Steve Kinney. The four coordination patterns (pipeline, manager, handoffs, fan-out) and the six failure modes. A clear minimalist view of the loop layer.
Loop Engineering, Cobus Greyling. The six-block framework (scheduling, worktrees, skills, plugins, sub-agent verifiers, durable memory). Useful complement to the three-controls framing.
I Don’t Prompt Claude Anymore. I Write Loops That Prompt Claude., James Fahey, Medium (June 2026). Carries the Cherny verbatim quote in full plus a maturity checklist for production loops.

The companions inside the series (all mapped in Section 7): Article 7bis is context engineering at single-document scope, Article 13bis is loop engineering at single-document scope, and Articles 17ter and 17quater lift both to corpus scope. The same pair recurs in the series’ follow-up work at every new scope: a context-engineering piece and a loop-engineering piece for document intents, for tool catalogues, and for the agentic case. Reading the two side by side, at whatever scope, is the fastest way to feel the boundary between them.