I'm trying to integrate Generative AI (ChatGPT, Gemini) into our CI/CD pipeline to automatically generate unit tests. Is anyone successfully using these models to write test cases for legacy codebases that lack documentation? I'm worried about the accuracy of the generated tests.
3 answers
Generating unit tests with Generative AI (ChatGPT, Gemini) is one of the most practical applications for modern software development. For legacy code, the model first needs to "explain" the code to you; once you verify its understanding is correct, you can prompt it to generate the corresponding test suites. In my current project, we used this method to increase our test coverage from 40% to 75% in just a few weeks. The key is to use the "Chain of Thought" prompting technique, where the AI describes the logic flow before writing the test code. This significantly reduces hallucinations and ensures the assertions are meaningful.
Kimberly, how do you handle security concerns when pasting proprietary legacy code into Generative AI (ChatGPT, Gemini) for test generation?
Make sure to check the generated mocks; the AI often makes assumptions about your database schema that might not be entirely accurate in reality.
Exactly, Karen. I always tell my team to treat the AI-generated mocks as a template. You still need a human eye to ensure the data structures align with the production DB.
Daniel, that is the biggest hurdle for most enterprises. We strictly use the "Enterprise" versions of these tools which promise that our data isn't used for training. For very sensitive modules, we use locally hosted versions of open-source models like Llama. This gives us the same generative power without the risk of leaking our intellectual property to the public cloud providers.