We love the results we get from , but the token costs are spiraling, especially with multiple agents reflecting on each other's work. Does anyone have tips on optimizing the 'verbose' mode or using smaller models like Llama 3 via Ollama for specific sub-tasks within the crew?
3 answers
You can definitely mix and match models in to save money. I run my 'Researcher' agent on a local Llama 3 instance using Ollama for the heavy lifting and data gathering, which costs zero tokens. Then, I only use GPT-4 for the 'Writer' or 'Manager' agents who need that extra level of reasoning. This hybrid approach reduced our monthly API bill by nearly 60 percent since we started this project in 2023. Also, try setting verbose=False for agents that don't need a public trail of their thoughts to save on output tokens.
Have you tried limiting the max_iter parameter for your agents? Sometimes they keep spinning their wheels trying to perfect a task when a "good enough" version was reached three iterations ago.
Caching is another lifesaver. has built-in caching for tools, so if your agents are asking the same questions, it won't hit the API again for the same result.
Caching definitely helps for repetitive research tasks. I noticed a huge jump in speed once I enabled it for my market analysis crew last month.
I haven't messed with max_iter yet! That’s a great idea. Is there a specific number you recommend for creative writing tasks versus technical coding tasks?