I am noticing high token consumption when using AutoGen (Microsoft Agent Framework) for long group conversations. What are the best practices for managing context windows and pruning conversation history so the agents don't keep processing irrelevant past messages while trying to reach a consensus on a specific task?
3 answers
To optimize costs in AutoGen (Microsoft Agent Framework), you should implement a custom context management strategy. One effective way is to use a "Summarizer" agent that periodically condenses the conversation history. Instead of passing the full transcript, you pass the summary and the last 3-5 messages. Additionally, you can set the max_round_count in your GroupChatManager to prevent infinite loops. Another tip is to use cheaper models like GPT-4o-mini for the manager and initial drafting, while reserving high-reasoning models only for the final verification node to balance performance and expenditure.
Have you tried using the 'filter' function in the GroupChat to dynamically select which agents receive certain parts of the history?
The most straightforward way is to clear the message buffer once a sub-task is completed and start a fresh context for the next phase.
True, but make sure the "Planner" agent keeps the high-level goals so the team doesn't lose sight of the project's overall objective after the buffer clear.
That is a great suggestion. By limiting the context to only the relevant "specialists" in the loop, you can shave off thousands of tokens per iteration, especially in a system with 5 or more agents.