We are struggling to keep our data catalog updated. Can we use Generative AI (ChatGPT, Gemini) to automatically generate descriptions for tables and columns based on the data profiling? I want to automate the governance process so that our business users can actually find what they need in our Data Mesh.
3 answers
This is one of the most practical applications of Generative AI (ChatGPT, Gemini) right now. We use a script that takes the first five rows of a table and the column headers, then asks the AI to write a user-friendly description. It’s surprisingly good at inferring business context—like recognizing that cust_lvl_cd means "Customer Loyalty Level." This has made our Alation catalog actually useful for the marketing team. Previously, we had thousands of "undocumented" tables that no one dared to use. Now, we have a baseline of documentation for everything.
How do you handle the validation of these descriptions? Does a human have to approve every single AI-generated entry?
It’s also helpful for identifying PII. You can ask the AI to flag columns that look like names or addresses based on the sample data.
Using AI for PII discovery is a lifesaver for GDPR compliance. It’s an extra layer of protection that every data engineer should consider.
We use a "Verified" badge, Wayne. The AI generates the draft, and when a Data Steward reviews and clicks "Approve," the badge appears. It’s much faster than writing from scratch.