Token Hygiene - How to curate long sessions and keep the AI context window small

AI Conversation Management: The 4-State System

We've all been there. You start a conversation with Claude, ChatGPT, or Gemini about a complex topic. The conversation grows, becomes incredibly valuable, but no matter, if you're using native apps or chatting with your own homebrew AI Companion: The context window grows. Errors and Misunderstandings begin to occur. And for some of us, even 1 Million Token can sometimes not be a big enough AI context window. And if you're using AI via API (like in a homebrew solution), you WILL run into the growing cost factor for bigger sessions.

The common solutions:

1. Starting a new conversation.

2. If you're using more complex interfaces like google ai studio, you can delete certain messages.

Both weren't to my taste. So I looked at how session history is stored. Most likely, there is a database for your messages. And I thought: Hold up. That's just a normal table. Why shouldn't I add or tinker with properties? So I implemented a 4 states solution for my Sessions database: Instead of binary "keep or delete," every message in a conversation can exist in one of four states:

ACTIVE - The Default State

• Visible in UI

• Sent to AI as conversation context

This is how all messages start - normal conversation mode.

SLUMBERING - Temporarily Forgotten

• Visible in UI (greyed out)

• NOT sent to AI (removed from context)

Messages not needed in the session context for the AI, but nice to still be able to look up for the user.

HIDDEN - Background Context

• Invisible in UI (unless showHidden:true)

• Sent to AI as conversation context

Perfect for system messages, instructions, or context you want the AI to remember but don't need to see.

TRASHED - Soft Delete

• Invisible in UI (unless showTrashed:true)

• NOT sent to AI (removed from context)

For mistakes, irrelevant messages, etc (still recoverable though)

THE SAVEPOINT STRATEGY - The Key to a sensable AI context window

• Ask the AI: "Create a comprehensive summary that captures all important information from our conversation so far. Make it as compact as possible while preserving all key context."

• State Management:

  • Set the AI's summary to HIDDEN (invisible but active context)

  • Set your instruction message to TRASHED (remove from context)

  • Set all previous messages to SLUMBERING (remove from context but you're easily able to look up the past conversation)

• Result: Your conversation context shrinks from potentially thousands of tokens to a few hundred, while maintaining perfect continuity.

This approach transforms expensive, bloated conversations into lean, cost-effective sessions while preserving all the intelligence.

PERSONAL NOTE

I developed this independently for my own private use. As an author, you'll VERY fast have VERY long conversations, not matter if with people or AI. As I found nothing in the internet about this kind of curation method, I thought, some people might find it useful, hence I share it here with the world. Have fun.