Context Engineering: How to Give an LLM the Right Information

Context engineering is the work of deciding what an LLM sees on each call: which instructions, documents, tool results and past messages go into the input, in what order, and how much of each. The model only knows what is in its context window plus what it learned in training. If the right fact is missing, buried or drowned out by noise, even a strong model gives a weak answer.

Think of a new colleague handling a customer call. A good manager does not hand them the whole company wiki. They give them the refund policy, the customer’s order and the last few messages. Context engineering is doing that job for the model, every time it is called.

Context engineering vs prompt engineering

	Prompt engineering	Context engineering
Focus	How the instructions are worded	Everything the model receives on a call
Typical work	Clear task, examples, output format	Retrieval, history, tool results, memory, ordering, trimming
Changes when	You rewrite the prompt	Every call, because the data changes
Main risk	Vague or conflicting instructions	Missing facts, stale facts, too much noise, cost

Prompt engineering is part of context engineering: the instructions are one piece of the context. The term became common once teams moved from single prompts to agents and multi-step apps, where most of the input is data gathered at run time, not text a person wrote. See prompting and structured output for the instruction side.

What goes into the context

Every call is built from some of these parts, and all of them share one token budget (see context windows and hallucinations):

• Rules: the system prompt. Who the model is acting as, what it may and may not do, the output format.
• The task: the user’s current question or the agent’s current step.
• Retrieved knowledge: passages from documents, usually found with search or RAG.
• Tool results: output from database lookups, APIs or code the model asked to run (see tool calling).
• History: earlier messages in the conversation.
• Memory: facts saved from earlier sessions, such as a user’s preferences (see agent memory and planning).
• Room for the answer: tokens you keep free so the reply is not cut off.

Build the context on purpose

The example below builds the input for a support assistant. It always includes the rules and the task, then adds the tool result the step depends on, then documents above a relevance cut-off, and finally the newest conversation turns that still fit. It keeps 40 tokens free for the answer:

def count_tokens(text):
    """Rough count: about 4 characters per token for English. Use your model's tokenizer for real counts."""
    return max(1, len(text) // 4)

def build_context(task, rules, history, documents, tool_results, budget, reserve_for_answer):
    """Assemble the model input in a fixed order, giving each part only what it needs."""
    available = budget - reserve_for_answer
    parts = [("rules", rules), ("task", task)]           # always included, kept short
    used = sum(count_tokens(t) for _, t in parts)

    # Tool results the current step depends on come next.
    for text in tool_results:
        if used + count_tokens(text) <= available:
            parts.append(("tool", text)); used += count_tokens(text)

    # Then the most relevant documents, best first.
    for score, text in sorted(documents, reverse=True):
        if score >= 0.5 and used + count_tokens(text) <= available:
            parts.append(("doc", text)); used += count_tokens(text)

    # Recent turns last: keep the newest ones that still fit, oldest are dropped.
    recent = []
    for turn in reversed(history):
        if used + count_tokens(turn) > available:
            break
        recent.insert(0, turn); used += count_tokens(turn)
    parts += [("history", t) for t in recent]
    return parts, used

rules = "Answer only from the documents and tool results. If the answer is not there, say you do not know."
task = "Customer asks: can I still return order 5521, and when would I get my money back?"
history = [
    "Customer: Hi, I bought a kettle last month.",
    "Agent: Thanks, could you share the order number?",
    "Customer: It is 5521.",
]
documents = [
    (0.92, "Items can be returned within 30 days of delivery if unused and in the original box."),
    (0.89, "Refunds are paid within 14 days of the return arriving at the warehouse."),
    (0.40, "Gift cards cannot be exchanged for cash."),
]
tool_results = ["order_lookup(5521): delivered 2026-10-12, item: kettle, status: delivered"]

parts, used = build_context(task, rules, history, documents, tool_results, budget=150, reserve_for_answer=40)
for kind, text in parts:
    print(f"{kind:<8} {count_tokens(text):>3}  {text[:60]}")
print(f"total {used} tokens of 110 available (40 kept for the answer)")

rules     24  Answer only from the documents and tool results. If the answ
task      20  Customer asks: can I still return order 5521, and when would
tool      18  order_lookup(5521): delivered 2026-10-12, item: kettle, stat
doc       20  Items can be returned within 30 days of delivery if unused a
doc       18  Refunds are paid within 14 days of the return arriving at th
history    5  Customer: It is 5521.
total 105 tokens of 110 available (40 kept for the answer)

What happened:

• The gift card passage was left out because its relevance score was low. It would only distract the model.
• The two oldest messages were dropped. The order number they led to is already in the tool result, so nothing useful was lost.
• The model now has exactly what it needs: the delivery date, the 30-day return rule and the 14-day refund rule.

The budget here is tiny so the trimming is easy to see. Real windows are far larger, but the same choices apply, because long inputs cost more, run slower, and models tend to miss details in the middle of a long context.

Trim tool results before they reach the model

APIs often return far more than a step needs. Passing the whole response wastes tokens and can leak data the model should never see:

import json

def count_tokens(text):
    return max(1, len(text) // 4)

raw = {
    "order_id": 5521, "status": "delivered", "delivered_at": "2026-10-12",
    "items": [{"sku": "KT-200", "name": "kettle", "qty": 1, "price": 39.0}],
    "warehouse": "BLR-2", "carrier": "FastShip", "tracking": "FS99812231IN",
    "billing_address": {"line1": "12 MG Road", "city": "Bengaluru", "pin": "560001"},
    "payment": {"method": "card", "last4": "4242", "auth_code": "A81723"},
    "audit": [{"at": "2026-10-09T10:12:00Z", "event": "created"},
              {"at": "2026-10-10T08:01:00Z", "event": "packed"},
              {"at": "2026-10-12T15:44:00Z", "event": "delivered"}],
}

def for_the_model(order, fields):
    """Pass on only the fields the current step needs, in a compact form."""
    return json.dumps({k: order[k] for k in fields}, separators=(",", ":"))

full = json.dumps(raw)
slim = for_the_model(raw, ["order_id", "status", "delivered_at"])
print("full tool result:", count_tokens(full), "tokens")
print("slim tool result:", count_tokens(slim), "tokens ->", slim)

full tool result: 134 tokens
slim tool result: 16 tokens -> {"order_id":5521,"status":"delivered","delivered_at":"2026-10-12"}

The slim version is about eight times smaller, and the address and payment details never enter the prompt. In an agent that calls tools twenty times, this is the difference between a focused context and one full of noise.

Techniques that work

1. Fixed layout. Keep the same order on every call: rules, task, tool results, documents, history. Stable layouts are easier to debug and let providers cache the unchanging start of the prompt.
2. Retrieve, don’t paste. Fetch the few passages that answer this question instead of including whole documents.
3. Summarise old history. Replace early turns with a short summary of decisions and facts, and keep the last few turns word for word.
4. Shape tool output. Return only needed fields, and cap list lengths (“first 20 rows of 3,400”).
5. Write things down outside the context. Long-running agents can save notes or a to-do list to a file or store and read back only what the next step needs.
6. Split the work. Give a sub-task to a separate call with its own clean context, and pass back only the result.
7. Measure. Log what was in the context for every call, and test with questions whose answers are in, and not in, your data.

Common mistakes

1. Using the whole window because it is there. More text raises cost and latency and can lower accuracy.
2. No room left for the answer, so replies stop mid-sentence.
3. Stale facts. A cached document or an old summary that contradicts current data.
4. Conflicting instructions spread between the system prompt, retrieved text and tool descriptions.
5. Not logging the final context, so nobody can tell why the model answered as it did.

What interviewers look for

When asked “what is context engineering?” or “how do you manage context in an agent?”:

• A one-line definition: choosing what the model sees on each call, within a token budget.
• The parts of a context and how prompt engineering fits inside it.
• Concrete techniques: retrieval, history summaries, trimming tool output, sub-agents, memory outside the window.
• How you would test it: logged contexts, answerable and unanswerable test questions, cost and latency tracked per call.

Practise with the LLM interview questions and the AI engineer interview questions.

Key takeaway

Context engineering decides what an LLM sees on every call: rules, task, retrieved documents, tool results, history and memory, all sharing one token budget. Choose the few pieces the step needs, keep a fixed order, trim tool output and old history, and leave room for the answer. Next: choosing a model.