Case Study 2.2: Elena and the Vanishing Memory

Understanding Context Windows in Practice

Background

Elena is a freelance strategy consultant who has been using AI tools as a core part of her practice for over a year. Unlike many AI users, Elena has invested time in understanding how to work with these tools well. She structures her prompts carefully, maintains project notes that she can inject into conversations, and has developed a consistent workflow for document-based work.

Despite this sophistication, she ran into a problem that took her months to diagnose properly — a problem that was not caused by poor prompting or bad technique, but by a fundamental property of how language models work.

The engagement that made it visible was a three-month strategy project for a professional services firm. The project involved synthesizing a large amount of interview data, structuring a multi-part strategic recommendation, and producing a series of deliverables — from a diagnostic summary to an implementation roadmap. Elena used AI tools heavily throughout, both for document drafting and for working through strategic questions in an extended conversational format.

The Pattern She Noticed

About six weeks into the project, Elena began noticing something strange. She was working through the implementation section of the main deliverable, using an AI tool to help develop and pressure-test the recommendations. The AI's suggestions were often good, but they kept missing a critical constraint.

The client had a specific structural limitation: they could not implement any changes that required modifying their core technology platform for at least eighteen months due to an ongoing migration project. This constraint had been established clearly in the very first AI conversation on the project — a long session where Elena had input the interview synthesis and walked through the initial strategic findings. The model had explicitly acknowledged the constraint multiple times in that session.

Now, weeks later, the model was recommending approaches that directly required platform modifications. When Elena pointed this out, the model apologized, acknowledged the constraint, and adjusted the recommendation. Two exchanges later, it made the same type of recommendation again.

Elena initially attributed this to inconsistency in the model. She tried rephrasing her instructions. She tried being more emphatic. The problem persisted.

The Diagnosis

Elena had a conversation with a technically sophisticated colleague who asked her a pointed question: "How long are these conversations getting, and how are you starting new sessions?"

She had been working in very long sessions — sometimes four to five hours of continuous conversation, with hundreds of exchanges. And she had not been re-providing the project constraints at the start of each session. She had assumed — reasonably, from a human perspective — that the model would "remember" what had been established.

Her colleague explained context windows. The model did not have persistent memory. Each session started fresh. And within long sessions, the finite context window meant that content from hours earlier — including the constraint document she had carefully constructed at the start — was no longer visible to the model by the time she was deep in the tactical details.

Elena tested this hypothesis. She started a new session on the project, this time without any re-provided context, and asked the model about the strategic approach. It made platform modification recommendations immediately. She then started another session with the constraint document re-provided at the top and asked the same question. The constraint was correctly respected.

The hypothesis was confirmed. The model had not been "forgetting." It had never had access to the early context in the first place — because that content had rolled out of the context window.

Why This Was Easy to Miss

There are several reasons this problem is easy to miss and hard to diagnose without knowing about context windows.

The symptoms look like model inconsistency. When a model gives advice that ignores constraints it previously acknowledged, the natural interpretation is that the model is being inconsistent or unreliable. The more precise interpretation — that the model literally cannot see content that has left the context window — is not obvious without understanding the mechanism.

The interface shows everything. Elena could scroll up in her conversation history and see the constraint document she had input at the beginning. From her perspective, it was part of the conversation. She had no reason to know that the model's view of the conversation was different from hers — that the interface was showing her the full history while the model was only processing the recent portion.

The problem is intermittent. In shorter sessions, the problem did not occur. The constraint document was within the context window and the model respected it. Elena's work on this project had gradually escalated in session length as the work became more detailed, and the context window problem only emerged at a certain scale. There was no sharp transition — just a gradual degradation that looked like ordinary inconsistency.

The model fills the gap plausibly. When the constraint document is no longer in context, the model does not produce obviously broken output. It produces strategic recommendations that are internally coherent and well-reasoned — they just happen to violate the constraint it no longer knows about. The output quality looks fine; only the constraint violation is a problem.

The Full Impact

When Elena traced the problem through her project work, she found it had affected more than just the platform constraint issue.

In one instance, she had established early in a session that the client's primary audience was mid-level managers rather than C-suite executives. Later in the same session, she had asked for communication recommendations, and the model had suggested a communication strategy calibrated for executive audiences. She had accepted it without noticing the misalignment, had drafted materials based on it, and had received feedback from the client that the tone felt off for their actual audience.

In another instance, she had established specific vocabulary preferences — terms the client had explicitly rejected as creating friction with their team culture. Later in a long session, those terms had reappeared in model output because the vocabulary guidance was no longer in the context window. The model was not being careless; it simply could not see the guidance anymore.

The cumulative effect of context dropout across a long project was not catastrophic, but it was significant: repeated re-explanation of constraints to the client, minor rework on documents, and an erosion of her own confidence in the tool during a critical project phase.

What Elena Rebuilt

Elena overhauled her working practices significantly after understanding context windows. Her new approach involves three structural changes:

The Project Context Document. For every project, Elena now maintains a structured context document that contains: the client's core constraint set, key vocabulary preferences and prohibitions, established strategic decisions, and a brief summary of the project's current state. This document is rarely longer than 500 words. It takes fifteen minutes to maintain after each major session.

She begins every AI session — even if she is picking up work from an hour earlier — by pasting this context document. The overhead is small; the benefit is significant.

Session Length Limits. Elena now treats the context window as a working memory budget. For complex projects, she caps sessions at approximately 30-40 exchanges before starting a new session with fresh context. She has found that shorter, well-contextualized sessions produce more consistent output than long sessions where context gradually degrades.

Constraint Verification Checkpoints. At natural breakpoints in a working session — when moving from one section of a document to another, or from one phase of a problem to the next — Elena explicitly re-states the key constraints relevant to what she is about to work on. This adds a few sentences of overhead per transition and substantially reduces constraint drift.

The Deeper Lesson

Elena's experience illustrates a principle that runs throughout effective AI collaboration: the model does not maintain a model of your project. It does not have a persistent understanding of who you are, what you are working toward, or what constraints you have established. Every piece of context it can access must be in the current context window.

This is categorically different from how human collaboration works. When you work with a human colleague over weeks, they build up a richer and richer understanding of the project, your preferences, and the constraints you operate within. You can refer to decisions made weeks ago with a shorthand reference and they will remember. You can rely on them to flag when a new suggestion conflicts with a standing constraint.

An AI tool cannot do any of these things. The context window is everything it knows about your project in this moment. When content leaves the window, it might as well never have existed.

This is not a criticism of the technology — it is a design constraint that follows from the current architecture of language models. The appropriate response is not frustration but adaptation: managing context deliberately, as a skill, in the same way you manage any other resource that is limited and consequential.

Elena now thinks of context management as analogous to managing working memory in a complex project. You would not expect a human working memory to hold every detail of a multi-month project simultaneously — you would use external systems, notes, and summaries. The context window is the model's working memory. You are responsible for what goes into it.

What Changed for Elena's Clients

One visible benefit of Elena's revised practices was a notable improvement in the consistency of her AI-assisted output over the course of long projects. Documents drafted in later project phases more consistently respected framing and vocabulary established early. Strategic recommendations more reliably adhered to client constraints. The revision cycles shortened.

She also found that maintaining the context document had benefits beyond AI sessions: it served as a running record of established decisions that was useful for her own reference during complex projects. The discipline of maintaining the document made her thinking more organized and explicit than it had been before.

The problem that prompted the change — the vanishing memory — became, in retrospect, a forcing function for a practice that improved her work well beyond the specific issue it solved.

Discussion Questions

1. Elena's context document is 500 words or fewer. Why is length discipline important here, given what you know about context windows and token costs?

2. What are the most important categories of information to include in a project context document? What should be left out?

3. How does the context window limitation change the way you would structure a multi-session AI-assisted project? What information management practices become necessary?

4. Elena notes that the problem "looks like model inconsistency." How would you explain the distinction between genuine model inconsistency and context window dropout to a colleague who is frustrated by AI "forgetting" things?

5. Are there AI tool configurations or features (such as persistent memory or retrieval-augmented generation) that address the context window problem? What are their limitations?