Our company is moving toward a Data Mesh to resolve our central data engineering bottleneck. We want to treat 'Data as a Product' where each business unit owns its pipelines. How are you handling the 'Federated Governance' aspect without creating 50 different versions of a 'Customer' table?
3 answers
The shift to Data Mesh is 80% cultural and 20% technical. To prevent the "duplicate table" nightmare, you must establish a Central Governance Guild that defines global standards (like naming conventions and PII masking) while letting teams choose their own local tools. We implemented a "Global Data Catalog" where every domain must register their output ports. If the Marketing team wants to create a Customer table, they first have to check the catalog to see if the Sales domain has already produced a "Gold Standard" version. This "Federated" approach ensures autonomy without sacrificing a single source of truth for core entities.
How are you planning to incentivize the individual business units to actually maintain their data products once the initial excitement of the migration wears off?
Focus on the 'Self-Serve Platform' layer first. If the central team provides easy-to-use templates for Airflow or Terraform, the business units are much more likely to follow the global standards.
Spot on, Laura. If you make the "right way" the "easy way" through automation templates, the federated governance almost takes care of itself.
That's the toughest part, Kevin. To answer that, we tied "Data Product Health" to the department's quarterly KPIs. We use automated data quality checks (like Great Expectations) that report to a public dashboard. If a domain’s data freshness or accuracy drops, it’s visible to leadership. This moved data from being a "tech chore" to a "business asset," ensuring that the domain experts actually care about the quality of the pipelines they are now responsible for.