When should you start a new AI chat, and when should you continue?

A small, practical question that hides a big design decision. Every long conversation with an AI assistant is a balance between context (helpful) and clutter (degrading). This article gives a working rule of thumb.

The hidden cost of a long chat

Every message you send is processed against the entire prior conversation. That’s a feature — it’s what lets the model remember your style, your project, your earlier instructions. It’s also a tax: the more tokens behind the scrollbar, the more attention the model has to spread across them, and the more likely it is to surface a stale detail at the wrong moment.

The symptom is easy to recognize: the longer a conversation runs, the worse the answers get. Not all at once — gradually. The model starts giving generic answers, forgets instructions you set at the start, sometimes contradicts what it said an hour earlier. It isn’t getting lazy or tired: it’s simply running out of room. I see three mechanisms at work.

The whiteboard limit

Picture the model’s working memory as a small whiteboard. Everything in the conversation gets written on it: your messages, its answers, every file you attach. That whiteboard is the context window, and its size is strictly limited. Once it’s full, the oldest lines get wiped to make room for new ones. The model doesn’t forget the way a human forgets: your earliest instructions are simply no longer on the board.

Recency bias

Even before the board fills up, attention isn’t spread evenly. The model gives far more weight to what you just said than to what you said twenty messages ago. Strict rules set in the first message can quietly stop being applied by the fifteenth — not because the model decided to ignore them, but because they sit too far back to keep pulling weight.

The clutter effect

Every message takes up space on the board, including the ones you’d rather forget: the typos, the rejected drafts, the tangents, every let’s start that again. Nothing disappears. It all accumulates, competes with your actual goal, and drags the quality of the answers down.

A working rule

• Stay when the new turn depends on what was said three or four exchanges ago.
• Reset when the new turn is a different task wearing the same hat.
• Branch (a new chat seeded with a one-paragraph summary) when you want to keep momentum without the baggage.

Three habits make that rule easy to apply.

Start fresh, with a summary

When the model turns confused or generic, don’t insist: restarting beats forcing. Open a new chat and seed it with the best of the old one — a short summary of what you need, plus the strongest draft you produced so far. For example:

Here is a summary of what I need: [summary]. Here is the best draft we have so far: [text]. Let’s revise the second paragraph using these rules.

The first response of a fresh, well-seeded chat is almost always sharper than the fifteenth of a cluttered one.

Ask for a recap along the way

If you genuinely need to stay in the same thread — it holds decisions that would be hard to reconstruct — ask the model to summarize everything agreed so far. That brings your most important rules back to the front of its attention:

Summarize the key decisions and rules we have established in this chat. We will use this summary as our reference going forward.

My favorite variant: ask for a handover summary, with everything a new conversation would need to pick up the work. Your seed paragraph writes itself, and the branch option described above becomes trivial.

One task per conversation

The golden rule. Don’t mix unrelated topics in the same thread: you were brainstorming a recipe and suddenly need an email to your boss? Open a new window. The tighter the thread stays, the longer it stays useful.

This fading memory isn’t a bug: it’s how these systems are built. Working well with AI means managing its attention, not testing its endurance. The people who get the most out of these tools aren’t the ones who run the longest conversations, but the ones who know exactly when to hit New chat.

To feel the difference, run the experiment once. The next time a conversation passes fifteen or so messages, watch for the moment the answers start missing the mark. When it comes, open a fresh chat, write a two-paragraph summary of what you need, and compare that first response with the last few from the old thread. The gap is usually all the convincing you need.