We are looking into using AutoGen to have one agent write code and another acting as a QA Engineer to find bugs and suggest fixes. Does this framework handle this "adversarial" loop effectively in a production environment without getting stuck in a loop?
3 answers
We implemented exactly this for our CI/CD pipeline last quarter. We set up a "Coder" agent and a "Reviewer" agent using AutoGen. The magic happens with the max_consecutive_auto_reply setting, which prevents the agents from looping infinitely if they can't agree. The "Reviewer" agent is prompted with our specific quality standards and it actually runs the code in a local environment to check for errors. If the test fails, the error log is passed back to the "Coder" to fix. It’s significantly reduced our initial bug count before a human even sees the PR. It’s a very natural fit for Quality Management.
Does the "Reviewer" agent actually look at edge cases, or does it just check if the code runs without crashing?
The code execution feature is the game changer here. It doesn't just "think" it works; it verifies it works.
Spot on, Laura. Verification is the core of Quality Management, and the sandbox in this framework is top-notch.
That depends on how you write the "System Message," Philip. We provide our Reviewer agent with a checklist of edge cases (null values, timeout limits, etc.). Because AutoGen allows for "Teachable Agents" that can remember past mistakes, our QA agent actually gets better at finding specific types of bugs that our team typically makes over time. It’s essentially a self-improving quality gate.