Our Product Owners spend a massive amount of time writing detailed user stories and acceptance criteria. We are looking to build a tool that takes high-level project requirements and generates these stories automatically. Does anyone have experience with the "Chain-of-Thought" prompting technique to ensure the AI follows the "As a... I want... So that..." format correctly while maintaining technical feasibility?
3 answers
We use a specific persona prompt: "Act as a Senior Business Analyst with 10 years of experience in FinTech." This context helps the AI understand the regulatory constraints of our user stories.
I implemented a similar system for a Scrum team in late 2023. The secret to success with "Chain-of-Thought" is to ask the model to first "think" about the technical dependencies before writing the story. My prompt structure is: 1. Analyze the requirement. 2. Identify the persona. 3. List the technical steps. 4. Finally, write the User Story. This multi-step reasoning prevents the AI from generating generic "fluff." We also found that feeding the model our "Definition of Ready" as a system instruction significantly improved the quality of the generated Acceptance Criteria.
How do you handle the potential for "hallucinated" technical requirements? If the AI suggests an acceptance criterion that isn't actually possible with your current tech stack, doesn't that create more work for the developers who have to correct it later in the grooming session?
William, that's a very fair point. To mitigate this, we don't treat the AI output as final. It’s a "First Draft" system. The PO reviews it, and then we have a specific "AI-Grooming" pre-check where a senior dev quickly flags anything technically impossible. Even with that manual check, it has reduced the story writing time by nearly 60%, allowing our POs to focus more on strategic vision.
Persona prompting is a game changer! When we told the AI it was a "Lead Quality Engineer," the acceptance criteria it generated for edge-case testing were surprisingly thorough.