Long conversations overflow the context window. Instead of dropping old messages, summarise them: keep the recent messages verbatim and fold older ones into a running summary.
Implement a class SummaryMemory(max_tokens, summarize, count_tokens=lambda s: len(s.split())):
add(role, text): append a message. If the verbatim messages now exceed max_tokens in total, remove the oldest messages until the total fits, but always keep at least the newest message. Then call summarize(previous_summary, removed_messages) once with all the removed (role, text) pairs, and store the string it returns as the new summary. (The previous summary starts as "".)messages(): the list to send to the model. If there's a summary, it comes first as ("system", "Summary: " + summary), followed by the verbatim messages in order.