Fit Chat History in the Context Window

Every chat app eventually hits the model's context limit. The usual fix: keep the system prompt, then keep as many of the most recent messages as fit.

Implement truncate_history(messages, budget, count_tokens):

• messages is a list of dicts like {"role": "user", "content": "..."}, oldest first. Roles are "system", "user" or "assistant".
• count_tokens(text) returns a token count. Use it, don't estimate.
• Always keep every system message.
• Then walk backwards from the newest message, keeping messages while the total stays within budget. Stop at the first message that doesn't fit, so the kept history has no gaps.
• Return system messages first, then the kept messages in their original order.

Example (one token per word)

msgs = [{"role": "system", "content": "be brief"},
        {"role": "user", "content": "hi there"},
        {"role": "assistant", "content": "hello"},
        {"role": "user", "content": "what is RAG"}]
truncate_history(msgs, 6, lambda t: len(t.split()))
# system (2) + "hello" (1) + "what is RAG" (3) = 6 tokens