Agent Memory and Planning: How AI Agents Remember and Plan

A language model has no memory between calls: everything it knows about the current task must be in the prompt. Agents need two kinds of memory to work well, plus a way to plan tasks that take many steps. This lesson covers both, with small examples you can run.

Short-term memory: the current task

Short-term memory is the conversation and the agent’s actions so far. Sending all of it every time is expensive and eventually overflows the context window. A common pattern keeps the latest turns word for word and folds older ones into a summary:

import re

# --- Short-term memory: keep recent turns in full, fold older ones into a summary -------------
def build_context(turns, keep_last=2):
    older, recent = turns[:-keep_last], turns[-keep_last:]
    summary = "; ".join(t.split(":", 1)[1].strip() for t in older if t.startswith("user"))
    return ([f"summary of earlier conversation: {summary}"] if summary else []) + recent

turns = ["user: my order A-1042 is late", "agent: checking A-1042", "user: I also want to change my address",
         "agent: noted", "user: and can you check the refund for B-2210?"]
for line in build_context(turns):
    print("context |", line)

# --- Long-term memory: facts saved across tasks, retrieved only when relevant ----------------
memory = [
    {"customer": "c-7", "fact": "prefers email, not phone calls"},
    {"customer": "c-7", "fact": "had a damaged item in March, given a 10% voucher"},
    {"customer": "c-9", "fact": "VIP account"},
]
def recall(customer, question):
    words = set(re.findall(r"[a-z]+", question.lower()))
    return [m["fact"] for m in memory if m["customer"] == customer
            and words & set(re.findall(r"[a-z]+", m["fact"].lower()))]
print("recalled:", recall("c-7", "Another damaged item arrived, what can we offer?"))

context | summary of earlier conversation: my order A-1042 is late; I also want to change my address
context | agent: noted
context | user: and can you check the refund for B-2210?
recalled: ['had a damaged item in March, given a 10% voucher']

The first part shows the trimmed context: two older requests became a one-line summary, and the last two turns stay complete. In a real system, a model writes the summary.

Long-term memory: across tasks

Long-term memory stores facts that matter later, such as a customer’s preferences or past issues. The key is to retrieve only what is relevant, exactly like RAG. The second part of the example searched one customer’s memories for words in the new question and found the earlier damaged-item case, but not the unrelated contact preference. Real systems use embeddings for this search.

Rules for long-term memory:

• Store facts, not whole conversations, and record when and where each came from.
• Scope memory to the right user, so one customer’s facts never appear for another.
• Let users see and delete what is stored about them, and keep sensitive data out.
• Expire or update facts that go stale.

Planning

For multi-step tasks, a reliable pattern is plan-then-execute: the model writes a short plan, the agent runs it step by step, and when a step fails, the model revises the remaining plan instead of starting over or giving up.

# --- Planning: write a plan first, run it step by step, and re-plan when a step fails -------------
def make_plan(goal):              # stand-in for a model writing a plan for the goal
    return ["find order", "check stock for replacement", "create replacement order", "email customer"]

def run_step(step, state):        # stand-in tools; the stock check fails the first time
    if step == "check stock for replacement" and not state.get("checked_other_warehouse"):
        state["checked_other_warehouse"] = True
        return False, "out of stock in main warehouse"
    return True, "done"

def replan(plan, failed_step, reason):  # stand-in for the model revising the remaining plan
    i = plan.index(failed_step)
    return plan[:i] + ["check stock in second warehouse"] + plan[i + 1:]

plan, state, log = make_plan("replace damaged item for order A-1042"), {}, []
i = 0
while i < len(plan) and len(log) < 8:                      # step limit
    ok, note = run_step(plan[i], state)
    log.append(f"{plan[i]}: {note}")
    if not ok:
        plan = replan(plan, plan[i], note)
        continue
    i += 1
print("\n".join(log))

find order: done
check stock for replacement: out of stock in main warehouse
check stock in second warehouse: done
create replacement order: done
email customer: done

When the stock check failed, the plan was revised to try the second warehouse, and the remaining steps continued. A written plan also makes the agent’s behaviour easier to review, test and explain.

Common mistakes

1. Sending the full history every time, which raises cost and confuses the model.
2. Storing everything forever in long-term memory, including outdated or sensitive facts.
3. Memory shared across users by mistake.
4. No re-planning, so one failed step ends the whole task, or endless re-planning without a step limit.

Key takeaway

Agents need short-term memory for the current task (recent turns plus a summary) and long-term memory for facts across tasks (retrieved only when relevant and scoped to the user). Plan-then-execute with re-planning makes multi-step tasks more reliable and easier to review. Next: multi-agent systems.