Our company is integrating Machine Learning into our customer support portal, and I’ve been assigned as the Lead Business Analyst. I’m used to traditional software requirements, but defining "accuracy" and "data quality" for an AI model feels very different. What specific elicitation techniques should I use to ensure our stakeholders and data scientists are on the same page regarding the project's success metrics?
3 answers
Eliciting requirements for AI requires a shift from functional steps to "Data-Driven Stories." You should start with "Impact Mapping" to identify exactly which business problem the AI is solving. Instead of asking what the system should do, ask what decisions the model should influence. Use "User Story Mapping" but include a specific section for "Data Constraints" and "Model Performance Thresholds." It is also crucial to conduct "Data Discovery" sessions where you review the availability and cleanliness of historical records with your data engineers before the development begins to avoid setting unrealistic goals for the ML model.
Have you already defined the "Minimum Viable Model" (MVM) or are you still trying to gather the full list of features for the final version?
Using "Prototyping" with real data samples is the best way to show stakeholders how the AI will behave in different scenarios.
I agree with Melissa. I once spent months on a BRD for an AI project only to find out the stakeholders hated the "correct" results because they didn't match their intuition!
Charles, we are still in the early discovery phase. I’m trying to figure out if we should focus on a simple chatbot first or go straight to a predictive model that suggests solutions to agents. My biggest hurdle is explaining to the VP that we can't guarantee 100% accuracy from day one. How do you usually document "probabilistic" requirements in a way that doesn't scare away the budget holders? They are used to software either working or not working, with no gray area in between.