Prompt Adherence: Making Models Do What You Actually Asked

S1 Why This Chapter Matters

The entire premise of context engineering is that you can control a language model's behavior by controlling what goes into its context. Every technique in this book — retrieval, system prompts, tool integration, memory — exists to put the right information in front of the model at the right time.

None of it matters if the model doesn't use what you gave it.

A model that ignores your retrieval results and answers from its training data has made your RAG pipeline pointless. A model that follows nine of your twelve formatting rules has made your system prompt unreliable. A model that correctly uses every retrieved document but fabricates one additional detail has made your citations untrustworthy. These are not hypothetical failures. They are the daily reality of deployed LLM systems, and they are the reason this chapter exists.

The degree to which a model's output actually serves everything in its context is what we call prompt adherence. It decomposes into two dimensions, and understanding why they are separate is the key insight of this chapter:

Instruction following asks: did the model follow all the instructions? Did it use the requested format? Stay within the specified scope? Obey every constraint? If your system prompt says "respond in three bullet points, cite sources inline, never recommend competitor products," instruction following is whether all three of those requirements were satisfied.

Groundedness (also called faithfulness) asks: are all the claims in the response supported by the context? This does not mean the response must parrot the context verbatim — the model can and should reason over what it was given, draw inferences, synthesize across documents. Groundedness means that every factual claim in the output should be traceable to the provided knowledge or to valid reasoning on top of that knowledge. A claim that comes from the model's training data rather than the context — however plausible — is a groundedness failure.

These two dimensions are in tension with each other, and this tension is what makes prompt adherence hard. If you optimize only for groundedness — penalizing every claim that isn't directly supported — the model learns to deflect too aggressively. It refuses to answer questions it could have answered through straightforward reasoning over the context, because the safest way to avoid unsupported claims is to say nothing. If you optimize only for instruction following — rewarding the model for producing a complete, well-formatted, responsive answer — the model learns to hallucinate. It fills gaps with plausible-sounding information from its training data, because the surest way to satisfy every instruction is to always have an answer, even when the context doesn't support one.

Getting prompt adherence right means holding both dimensions in balance: follow the instructions and stay grounded, and when the context doesn't support a response, say so rather than fabricate one.

Both dimensions improve as language models get better — but neither is fully solved by better models alone. You can also improve prompt adherence through the system you build around the model: how you construct the context, how you orchestrate retrieval, how you validate outputs, and how you train the model for your specific requirements. This chapter covers all of these.

We start by looking at what's actually in the context you're assembling — because the structure of your context is what creates both problems.

What's In Your Context

Every system that uses a language model assembles a context from multiple sources. Consider a retrieval-augmented generation (RAG) system — the architecture behind most enterprise LLM applications. When a user asks a question, the system does not simply pass the question to the model. It constructs a context:

• The system prompt — written by the development team, often weeks before any user touches the system — establishes the rules of engagement: persona, tone, formatting requirements, scope ("only answer questions about our products"), and constraints ("never recommend competitor products," "always cite your sources inline").
• The user's input arrives — a question, a task, a request. This is what the system exists to serve. The user may also add their own instructions or constraints beyond what the system prompt specifies.
• The retrieval layer activates. It searches an index of documents — product specs, policy pages, internal knowledge bases — and surfaces the most relevant results. These retrieved documents are injected into the context as the knowledge the model should use to answer.
• Conversation history, tool outputs, or other state may also be folded in, depending on the system's design.

All of these layers are assembled into a single context window and handed to the language model. The instructions live in the system prompt and the user input. The knowledge lives in the retrieved documents, tool outputs, and conversation history. Instruction following checks adherence to the first. Groundedness checks adherence to the second.

What counts as "good" prompt adherence depends on the system you've built. For a RAG system grounding a support chatbot, you want strict groundedness: the model should answer using only the retrieved documents, not its training knowledge. For a coding assistant, you might want the model to use both the code you provided and its general programming knowledge — but to strictly obey constraints like "use only the standard library." For a creative writing tool, you might actively want the model to draw on its training knowledge freely — but still follow structural instructions about length, tone, or format. The two dimensions are universal; the balance between them is system-specific.

S2 The Two Dimensions of Prompt Adherence

We've described two kinds of expectations your system places on the model: follow the instructions, and ground in the knowledge. These map onto two dimensions that the research community discovered and studied largely independently, and that fail for different reasons.

Instruction Following: Did it do what you asked?

The first dimension is instruction following. This is about whether the model's output reflects everything the instructions asked for. Did it use the requested format? Did it stay within the specified scope? Did it follow every constraint? For example: your prompt says "respond in exactly three bullet points; do not add an introduction." A response that gives three correct bullets but starts with "Here are the key points:" has failed instruction following — the content may be fine, but the format rule was broken.

Instruction following has been a central concern of the field for years. The trajectory is worth understanding briefly, because it explains both the capability and its limits:

• In 2020, GPT-3 showed that language models can follow instructions — but only if you provide examples (few-shot prompting). The capability existed but had to be elicited. [Brown et al., 2020]
• Instruction tuning (FLAN, T0) demonstrated that if you train on diverse instructions, models generalize to new instructions they haven't seen. Instruction following became a transferable skill. [Wei et al., 2022; Sanh et al., 2022]
• RLHF (InstructGPT) made models dramatically better at following instructions — but also optimized for general helpfulness, which sometimes conflicts with following your specific instructions. A model trained to always give a comprehensive answer will resist being told to say "I don't know." [Ouyang et al., 2022]
• Recent evaluation work (IFEval, ComplexBench, IFScale) has shown that simple instructions are largely solved, but as constraints multiply — ten rules, twenty rules, format and scope and tone and citations — even frontier models start dropping things. The problem didn't disappear; it moved to a harder regime. [Zhou et al., 2023; IFScale, 2025]

If you think of the instructions in your context as a set of requirements, then instruction following is recall over those requirements: what fraction of them are reflected in the output? A failure is a dropped requirement — a formatting rule ignored, a constraint violated, a scope boundary crossed. In the language of classification, these are false negatives: things that should have been in the output (or should have shaped it) and weren't.

Groundedness: Did it do it right?

The second dimension is groundedness — also called faithfulness. This is about whether the model's claims are supported by the knowledge in the context. Not verbatim repetition — the model can and should reason over what it was given, draw inferences, and synthesize across documents. Groundedness means every claim should be traceable to the provided knowledge or to valid reasoning on top of that knowledge. A claim that comes from the model's training data rather than the context — however plausible — is a groundedness failure.

This dimension was discovered somewhat separately from instruction following, and the history matters:

• Work on faithfulness in summarization (Maynez et al., 2020) first showed that a model can perfectly follow the instruction "summarize this document" and still produce a summary containing claims the document never made. Instruction following and factual faithfulness turned out to be different things. [Maynez et al., 2020]
• Hallucination surveys (Ji et al., 2023) and benchmarks like TruthfulQA established that this isn't an edge case — it's systematic. Models confidently produce false information, and the problem doesn't simply improve with scale. [Ji et al., 2023; Lin et al., 2022]
• Fine-grained factuality work (FActScore, FACTS) showed that holistic "is this grounded?" judgments aren't precise enough. You need to decompose the response into individual claims and verify each one against the source. This claim-level evaluation is now the foundation for measuring groundedness. [Min et al., 2023; Jacovi et al., 2024]

If you think of the knowledge in your context as the set of supported facts, then groundedness is precision over that context: what fraction of the output's claims are actually supported? A failure is an unsupported claim — a fabricated detail, a misattributed fact, a piece of training knowledge that leaked into a response that should have been grounded entirely in the provided documents. These are false positives: things in the output that shouldn't be there.

Why the Decomposition Matters

These two dimensions fail independently. A single response can ace one and fail the other — or fail both. If you don't distinguish between them, your debugging goes in circles: you tighten the grounding instructions and the formatting breaks; you lock down the formatting and the hallucinations creep back in.

The decomposition also structures everything that follows in this chapter. Measurement requires separate tools for each dimension: claim-level verification for groundedness, criteria-level testing for instruction following. Interventions target different layers of the system depending on which dimension is failing. Even training approaches use separate reward signals — one per dimension — because holistic "is this response good?" signals are too blurry to drive reliable improvement.

A natural question is: are instruction following and groundedness the only dimensions of output quality? No. Reasoning correctness, safety, fluency, coherence, and creativity all matter too. But those dimensions are primarily about the model's internal capabilities — you don't fix reasoning by changing the context; you fix it by changing the model. Instruction following and groundedness are the dimensions most directly shaped by what you put in — by how you engineer the context. That's what makes them context engineering problems, and that's why they're the focus of this chapter.

(This scoping is imperfect. Reasoning is influenced by context — chain-of-thought prompting works, after all. And sycophancy, where a model overrides both its instructions and its context to agree with a user, sits at the intersection of both dimensions. We note these overlaps honestly rather than pretending the boundaries are clean.)

S3 Why Both Problems Persist

A reasonable question at this point: if we know what instruction following and groundedness are, and we've been studying them for years, why do they still fail? The answer lies in what language models fundamentally are — and what the training process optimized them to do.

Language models are, at their core, next-token predictors trained on vast corpora of text. When you give a model a context and ask it to respond, it is not executing your instructions the way a program executes code. It is generating text that is statistically consistent with the patterns it learned during training — patterns about what plausible, helpful, knowledgeable responses look like. As Andrej Karpathy has put it, LLMs are "dream machines": their default mode is generation from learned priors, and your context is the opening scene that directs the dream. Adherence is whether the dream stays where you pointed it.

This distinction — between executing instructions and simulating a scenario where instructions are being followed — is subtle but fundamental. The model does not want to follow your instructions. It does not want anything. It has learned to produce outputs that look like instruction-following, because its training data is full of examples where instructions were followed. But when your specific instructions conflict with the statistical patterns that dominated its training — when you ask it to say "I don't know" instead of giving a plausible-sounding answer, or to ignore information it "knows" is relevant — the training priors push back.

This is precisely what happened in the post-training process that made modern models feel so capable. RLHF and its successors optimized models to be generally helpful: fluent, comprehensive, knowledgeable, willing to engage. These are genuinely valuable properties. But they create a structural tension with prompt adherence. A model trained to be maximally helpful will use everything it knows to give you the best answer — including knowledge from its training data that contradicts or supplements the documents you retrieved. From the model's perspective, this is good behavior. From your system's perspective, it is a groundedness failure.

The chip recommendation example makes this concrete. You are building a support chatbot for a company that sells Qualcomm chips. Via retrieval, you surface the relevant Qualcomm product listings. A customer asks a general question about processor options. The model, drawing on its training, recommends an ARM chip — a competitor's product that appears nowhere in your retrieved documents. The model is not malfunctioning. It is doing exactly what it was trained to do: give a comprehensive, knowledgeable answer. It is just not doing what your system required.

This is also where the tension between the two dimensions becomes concrete. If you try to fix the groundedness problem by aggressively penalizing unsupported claims, the model starts deflecting too often — refusing to answer questions it could have handled through reasonable inference from the context. If you try to fix the instruction following problem by rewarding completeness and responsiveness, the model fills gaps with training knowledge rather than admitting uncertainty. Optimizing for one dimension at the expense of the other makes the system worse overall.

Research has made the root cause even more precise. Kalai et al. (2025) showed formally that standard training and evaluation procedures structurally reward guessing over admitting uncertainty. Models receive no reward for saying "I don't know," so they learn never to say it. Xu et al. (2024) went further, proving using computability theory that any sufficiently capable language model will inevitably hallucinate when used as a general problem solver — not as an engineering limitation, but as a mathematical property.

This does not mean the problem is hopeless. It means the problem is structural, not incidental, and will not be solved by simply waiting for better models. Every generation of models gets better at both instruction following and groundedness, but the underlying tension between general helpfulness and specific adherence persists. As Karpathy has observed about building AI products: getting a demo to work just requires the system to succeed on some inputs; getting a product to work requires it to succeed on all of them — and "every single nine is a constant amount of work." Your demo works because both dimensions happened to hold on the examples you tested. Your product fails because the dimensions fail independently, on inputs you did not anticipate.

This is why prompt adherence is an engineering discipline, not a feature request. Both dimensions improve with better models — but they also improve with better systems: better context construction, better orchestration, better validation. The remainder of this chapter gives you the tools for both.

S4 Recognizing Adherence Failures in Your System

Before you can measure or fix adherence, you need to recognize which dimension is failing. The taxonomy from the previous sections is not academic — it maps directly to patterns you will see in production. Each pattern below has a signature that tells you where to look.

Factual drift in RAG systems. The model supplements retrieved documents with knowledge from its training weights. This is a groundedness failure. It is particularly insidious because the response looks grounded — it has the right structure, it may even cite documents — but includes claims the documents never made. The tell: compare each factual claim in the response against the retrieved passages. If a claim has no source, the model is drifting.

Instruction erosion at scale. Your system prompt has fifteen rules. The model follows most of them most of the time. But as the context grows — more retrieved documents, longer conversation history, more tool outputs — adherence to specific rules degrades. This is an instruction following failure, and it is not random: research on position effects (Liu et al., 2024) shows that models attend most reliably to the beginning and end of their context, and least to the middle. Rules buried in the middle of a long system prompt are the first to be dropped.

Out-of-scope responses. A model built to answer questions about one product domain draws on general knowledge to answer questions outside that domain. This is a groundedness failure — the model reached past the knowledge you provided rather than declining to answer. The fix is not more instructions telling the model to stay in scope (though that helps at the margin). The fix is training the model to treat deflection as default behavior when the context does not support a response.

Conflicting context resolution. Retrieved documents contradict each other — because your knowledge base contains outdated versions, or because different sources describe different policies. The model resolves the conflict using its training priors rather than following explicit instructions about precedence. This is a failure of both dimensions: the model is not following the instruction about how to resolve conflicts (instruction following), and it is grounding in the wrong source or in no source at all (groundedness).

The pattern in all four cases: the model's behavior is governed by something other than what you put in the context. Your interventions should target the specific dimension that is failing. Tightening grounding instructions will not fix instruction erosion. Training for deflection will not fix formatting violations. The taxonomy tells you where to aim.

S5 Measuring Each Dimension

You cannot improve what you cannot measure — and in prompt adherence, how you measure determines whether you can act on the result. The single most important principle: measure groundedness and instruction following separately, using decomposition.

Why Holistic Scores Hide What's Failing

A common approach is to ask a judge model: "Is this response good?" and get a single score. This collapses both dimensions into one number. A response that scores 0.7 might be perfectly grounded but missing three formatting requirements, or it might follow every instruction but contain fabricated claims. The number tells you something is wrong. It does not tell you what, and it certainly does not tell you where to intervene. Decomposed measurement — one score for groundedness, one for instruction following — makes failures actionable.

Measuring Groundedness: Claim-Level Verification

Groundedness is precision over the context: what fraction of the output's claims are supported by the provided knowledge? The measurement technique follows directly from this definition.

Step 1: Decompose the response into individual claims. A response like "The Snapdragon 8 Gen 3 has a 30% faster GPU and supports Wi-Fi 7" contains two distinct factual claims. Each must be verified independently.

Step 2: Verify each claim against the source documents. For each claim, ask: is this entailed by the retrieved documents? A claim might be true in the world but not supported by the provided context — and for a grounded system, that counts as a failure.

Step 3: Aggregate. The fraction of supported claims is the groundedness score. A response where 9 of 10 claims are supported scores 0.9.

You can implement a basic version of this pipeline with prompting alone — ask a capable model to list the claims, then evaluate each against the source. For production use, purpose-trained evaluation models are more reliable: they are optimized specifically for the entailment judgment at Step 2, catching subtle failures where a claim is related to but not actually supported by the source document. The FACTS benchmark (Jacovi et al., 2024) is a useful stress-test: it pairs long source documents (up to 32K tokens) with requests that require grounded, long-form responses, and uses frontier LLM judges to evaluate both responsiveness and attributability.

Measuring Instruction Following: Natural Language Unit Tests

Instruction following is recall over requirements: what fraction of the prompt's requirements are reflected in the output? The measurement technique mirrors software testing.

Step 1: Decompose the prompt's requirements into individual, testable assertions. A system prompt with twelve rules generates twelve test cases. "Does the response use bullet points?" "Does it avoid mentioning competitor products?" "Does it include inline citations?"

Step 2: Evaluate each assertion against the response. Each test produces a score (pass/fail, or a continuous scale) and a rationale explaining the judgment.

Step 3: Aggregate. The fraction of satisfied requirements is the instruction following score.

This is the "natural language unit test" metaphor: each requirement becomes a test case, and the response either passes or fails each one. The technique was formalized in LMUnit (Kamalloo et al., 2024), which takes a prompt, a response, and a natural language test assertion, and returns a score with rationale. The practical value is diagnostic: instead of knowing that "something is wrong," you know that requirements 3, 7, and 11 are failing. Inter-annotator agreement improved from a Kappa of 0.04 with holistic preference judging to 0.52 with decomposed unit tests — a meaningful shift toward reproducible evaluation.

Finding the Breaking Points

Evaluation is only as useful as your test distribution is hard. A model that follows instructions reliably on simple examples may fail on the examples that matter most — the ones with conflicting requirements, long contexts, or edge cases that stress the model's training priors. Constructing progressively harder test cases — by adding constraints, introducing contradictions, or increasing context length — reveals the failure boundaries that easy examples hide.

One effective approach is synthetic generation of progressively harder test cases. Start by generating diverse natural language unit tests for each prompt, targeting fine-grained quality criteria. Then generate contrastive responses — responses that vary systematically in how well they satisfy each criterion, creating pairs that differ on exactly one dimension. This gives you test distributions where failure modes are isolated and identifiable. The key is to ensure coverage across failure types: not just easy formatting checks but subtle groundedness violations, multi-constraint interactions, and edge cases where the model's training priors conflict with the prompt's requirements.

S6 Improving Prompt Adherence in Practice

You have the taxonomy. You can diagnose which dimension is failing. You can measure each one separately. Now: what do you do about it?

The interventions exist on a spectrum from lightweight to heavy. We organize them as four levers, ordered by what you can control. Everyone can pull the first two. The third requires system-level control. The fourth requires training access. The right starting point depends on what you control and how systematic the failure is.

Lever 1: Model Selection

Your choice of base model matters more than most practitioners realize. Models differ significantly in their instruction-following and groundedness capabilities, and these differences are not captured by general benchmarks.

Two practical insights. First, if you are fine-tuning for adherence, start from an instruct model rather than a chat model. Chat models have undergone heavier post-training that bakes in more opinionated behavior — exactly the behavior you are trying to override. Instruct models sit earlier in the post-training pipeline and are more malleable. Starting from there gives the fine-tuning more room to shape the model toward what you need.

Second, some models are specifically trained for grounded generation — models designed to prioritize faithfulness to provided context over training knowledge. If your system's primary requirement is groundedness, choosing such a model is the single highest-leverage decision you can make. It is more impactful than any prompting technique or orchestration pattern.

Lever 2: Prompting

Prompt engineering has an adherence-specific toolkit that goes beyond general "write better prompts" advice. Three techniques are worth knowing:

Explicit grounding instructions. If the model is reaching outside the retrieved documents, stating explicitly — "Answer only using information from the provided documents. If the documents do not contain sufficient information to answer, say so." — can move the needle. This is the simplest intervention and the first thing to try.

Constraint ordering. Position matters. Research on positional attention (Liu et al., 2024) shows that models attend most reliably to the beginning and end of their context, and least to the middle. Put your most critical constraints first. Put your most commonly violated constraints early. If a rule keeps getting dropped, move it toward the top of the system prompt before you try anything else.

Positive framing. Negative instructions can backfire. "Do not use mock data" makes the model more aware of mock data as a concept, and in some cases more likely to produce it — an effect called the "pink elephant problem." The fix is to reframe as positive commands: "Use only real data from the provided sources." Tell the model what to do, not what not to do.

The ceiling on all prompting techniques is that they are brittle. They help at the margins. A well-crafted prompt can improve adherence from 85% to 92%, but it will not take you from 70% to 99%. When the model's training priors conflict strongly with your requirements, the prompt is fighting the weights — and the weights usually win.

One important caveat: research on instruction hierarchy (Geng et al., 2025) has shown that the separation between system and user prompts does not establish the reliable priority most practitioners assume. Models have inherent biases toward certain types of constraints regardless of where they appear in the prompt. Do not rely on the system/user distinction alone to enforce instruction precedence.

Lever 3: Inference-Time Orchestration

Prompt adherence is not just a model property — it is a system property. Every component in your pipeline has its own form of adherence to maintain. The reranker follows ranking instructions. The generator follows grounding and response instructions. Enforcing adherence at a single point is not enough; it must be trained and enforced at each layer.

Upstream: Instruction-aware retrieval. If your retrieved documents contain conflicts, stale information, or irrelevant results, you have made the groundedness problem harder before the generator even runs. An instruction-following reranker can enforce recency ("prioritize documents from the last six months"), source hierarchy ("internal documentation takes precedence over market reports"), and conflict resolution ("when sources disagree, prefer the most recent one") at the retrieval layer. This is adherence enforcement at the point where knowledge enters the context.

Downstream: Second-pass verification. Generate a response, then run it through a verification step before returning it. For groundedness: decompose the response into claims and check each against the source documents. Flag, regenerate, or remove unsupported claims. For formatting: if the model does not reliably produce structured output in a single pass, a lightweight extraction step as a fallback — asking a second call to pull structured content from the freeform response — is often more reliable than more elaborate prompting.

Second-pass approaches are often the practical ceiling for inference-time-only solutions. A well-designed pipeline with retrieval-layer enforcement and post-generation verification can handle a significant fraction of adherence failures without touching model weights.

Lever 4: Training

When failures are systematic — when the model's training priors conflict deeply with your requirements and no amount of prompting or orchestration overcomes them — you need to change the weights. This does not mean training from scratch. It means targeted fine-tuning, followed by reinforcement learning with the right reward signals.

Step 1: Supervised fine-tuning (SFT). The goal of the SFT step is to teach the model a structural distinction it does not learn in standard post-training: the difference between parametric knowledge (what it learned during pretraining) and non-parametric knowledge (what is in the provided context right now). Fine-tuning on a structured format that makes this distinction explicit — with inline attributions linking each claim to its source document — enables the deflection behavior that is central to grounded generation. A model that can distinguish "I know this from training" from "this is in the document I was given" can then be trained to prefer the latter, and to refuse to answer when neither suffices.

Step 2: Reinforcement learning with decomposed rewards. The critical insight for the RL step is that you need two separate reward signals, one for each dimension of prompt adherence.

• The groundedness reward operates at the claim level: decompose the response into individual claims, score each against the source documents using a purpose-trained evaluation model, and average. This directly targets precision — penalizing claims that are not supported by the provided context.
• The instruction following reward operates at the criteria level: extract the requirements from the prompt, score each as a natural language unit test against the response, and average. This directly targets recall — penalizing requirements that were dropped or violated.

Why decomposed rewards matter: A holistic reward signal ("is this response good?") is underspecified. Different evaluators focus on different qualities, producing inconsistent signal and slow learning. Decomposed rewards give the model specific, interpretable signal per dimension. The model knows which claims were unsupported and which instructions were violated — not just that "something was off."

The training lever is the heaviest to pull. But for systematic failures — the cases where the model's training priors are actively working against your requirements — it is the only lever that changes the model's default behavior rather than fighting it at inference time.

S7 What Happens at the Limits: Instruction Overload

Everything we have discussed so far assumes a manageable number of constraints. But what happens when your system prompt grows — ten rules, twenty, fifty?

The answer, established by recent empirical work (IFScale; Jaroslawicz et al., 2025), is that adherence degrades non-linearly as constraints multiply. And the pattern of degradation depends on the model architecture.

IFScale tested frontier models on instruction sets scaling from 10 to 500 constraints and found three distinct degradation patterns:

• Threshold decay (reasoning models like o3): adherence holds steady up to a point, then drops sharply. The model appears to handle complexity well until it hits a cliff.
• Linear decay (models like GPT-4.1, Claude Sonnet 4): adherence degrades gradually and predictably. Each additional constraint costs roughly the same amount of adherence.
• Exponential decay (models like GPT-4o, Llama-4-Scout): rapid collapse. Adherence degrades faster with each additional constraint.

In all three patterns, the study found a positional bias: models attend more reliably to instructions that appear earlier in the prompt. Later instructions are more likely to be dropped, regardless of importance.

This is not the same as context length degradation, where information in long contexts becomes less attended to as distance increases. Instruction overload is something more specific: instruction interference. Adding constraint #11 does not just strain capacity — it can actively displace adherence to constraint #9. The model's ability to simultaneously satisfy an expanding set of requirements degrades in ways that are not simply additive. You are not overflowing a buffer. You are overloading the model's ability to track and honor competing requirements.

The practical implications are significant:

1. You cannot simply add more rules. At some point, each new instruction trades off against existing ones. System prompt design requires prioritization, not just accumulation.
2. Know your model's degradation pattern. If you are using a model that exhibits threshold decay, test to find where the cliff is. If linear, budget a per-constraint cost. If exponential, keep constraints minimal and offload enforcement to orchestration.
3. Position your most critical constraints first. This is supported by both the IFScale positional bias finding and the "Lost in the Middle" attention research. The beginning of your system prompt is prime real estate.

To make these patterns concrete, here are selected results from IFScale at key constraint densities (accuracy %):

The error analysis reveals an important detail: at moderate constraint densities (150-200), later instructions fail at roughly twice the rate of earlier ones — a strong primacy bias. At extreme densities (300+), this bias diminishes as the model shifts from selective instruction dropping to near-universal abandonment. Reasoning models like o3 also behave differently in failure: they attempt morphological approximations of dropped instructions rather than simply omitting them, suggesting they "try harder" before failing.

S8 Composition — How the Levers Stack

The four levers from S6 are not alternatives. In practice, the most robust systems compose them. The progression is instructive:

A general-purpose model with careful prompting might achieve 70-80% adherence on your requirements. Adding retrieval-layer enforcement and second-pass verification pushes that to 85-90%. Supervised fine-tuning on a structured format that distinguishes parametric from non-parametric knowledge can move groundedness from that baseline into the low 80s. And adding RL with decomposed rewards — one for groundedness, one for instruction following — pushes further still. The final ingredient, inference-time chain-of-thought optimization, can yield additional gains.

As a concrete example: on the FACTS benchmark for grounded generation (Jacovi et al., 2024), a pretrained Llama 3 baseline scores under 70% groundedness. SFT on a structured format that makes the parametric/non-parametric distinction explicit raises this to the low 80s — a gain of over 10 points from format alone. Adding RL with decomposed rewards pushes into the high 80s. Adding inference-time chain-of-thought pushes further still, reaching performance competitive with the strongest frontier models. Each step compounds because each targets a different aspect of the adherence problem — structural format, reward signal, and inference-time reasoning.

The broader pattern: deflection as default behavior is the hallmark of a well-composed adherence system. A general-purpose model treats every question as an invitation to be helpful. A model trained for grounded generation treats the absence of supporting context as a reason to decline. This is not a capability limitation — it is a trained design choice. The model has learned that "I don't have enough information to answer that" is the correct response when the context does not contain the answer, even if its weights contain a plausible-sounding one.

Inline attribution is the other compositional design choice worth highlighting. When citations are embedded directly in the response as it is generated — each claim linked to the source document that supports it — grounding becomes auditable. You do not need to trust that the model stayed within the context. The system surfaces exactly where each claim came from.

The stakes for getting composition right are real. In 2024, Air Canada's customer support chatbot told a customer they were eligible for a bereavement discount — a promise the company's current policy did not support. The chatbot had access to the correct policy document. It hallucinated anyway, reaching into its training priors instead of staying grounded. The company was held liable. More systematic evidence comes from a Stanford study (Magesh et al., 2025) finding that RAG-based legal tools hallucinate 17-33% of the time despite vendor claims of high accuracy. These are not individual model failures. They are failures of systems that lacked the compositional adherence infrastructure this chapter describes.

S9 Open Problems

The prompt adherence problem expands as the systems we build grow more ambitious. Three directions are worth watching, each connected back to the taxonomy.

Corpus-Scale Groundedness

As context windows grow — and as systems begin attending to entire corpora rather than retrieved snippets — the definition of groundedness expands with them. What manifests as hallucination in a RAG application becomes factual consistency across a corpus. Memory and personalization systems surface this challenge clearly: if a model has access to a long history of interactions, staying grounded in what the user has actually said — and not confabulating details from across that history — is groundedness at a new scale. Larger context means a larger surface for precision failures.

Deterministic Enforcement

Some constraints should not be probabilistic. A legal document generation system that follows the formatting rules 98% of the time still produces errors 2% of the time — and for some applications, that is unacceptable. Architectures that combine probabilistic model behavior with deterministic functions — where the model handles flexible reasoning and deterministic components enforce specific rules — represent a promising direction for constraints that must be satisfied with certainty, not with high probability.

Structured State for Agents

As agentic systems grow more sophisticated, adherence becomes a multi-step problem. An agent executing a ten-step task must maintain adherence to its instructions across all ten steps — and the context at step seven may look nothing like the context at step one. Explicit, structured state — data structures the agent reads from and writes to, rather than relying on the attention window to track the plan — is emerging as adherence infrastructure for long-horizon tasks. The model's context is no longer just the information needed to answer a question; it is a workspace where the state of a complex task lives. Making that state explicit reduces the degree to which the model has to rely on its own unreliable memory of what it was supposed to do.

S10 Summary

Prompt adherence is the degree to which a model's output serves everything the context demands. It decomposes into two independently-failing dimensions:

• Instruction following — recall over the requirements in your context. Did it do what you asked? Failures are dropped requirements: formatting rules ignored, constraints violated, scope boundaries crossed.
• Groundedness — precision over the knowledge in your context. Did it stay within what it was given? Failures are unsupported claims: fabricated details, misattributed facts, training knowledge leaking into responses that should be grounded entirely in the provided documents.

These dimensions fail for a structural reason: language models are trained to be generally helpful, comprehensive, and confident. This is in direct tension with systems that need narrow, constrained, uncertain-when-appropriate behavior. The tension does not resolve with scale.

The response is layered:

1. Measure each dimension separately — claim-level verification for groundedness, natural language unit tests for instruction following.
2. Improve using the lever matched to your failure — model selection, prompting, inference-time orchestration, or training with decomposed reward signals.
3. Compose the levers — the most robust systems enforce adherence at every layer of the pipeline, from retrieval to generation to verification.

The goal is not a model that never fails. The goal is a system whose failure modes are known, measurable, and bounded — and a mental model that lets you recognize and address them as they evolve.