Our team has been running a classic Medallion (Bronze/Silver/Gold) setup on Databricks for two years. However, with the rise of Apache Iceberg as a universal standard, we’re seeing a push to unify our batch and streaming layers into a single "Instant Lakehouse." Does the Medallion pattern still hold up when you need sub-second latency for AI agents, or is it becoming an ETL bottleneck?
3 answers
The Medallion pattern isn't dying; it’s evolving. In 2026, the "Gold" layer is no longer just for BI dashboards—it's the feeding ground for Context Engineering. We’ve kept the Medallion structure but replaced the heavy Spark transformations in the Silver layer with Streaming SQL using Flink. This allows our "Gold" tables to update in near real-time. The biggest shift we've seen is moving away from proprietary formats to Apache Iceberg, which lets our Snowflake and Databricks clusters read the same data without moving it. This "Zero-Copy" architecture is the real 2026 trend, not just a name change.
How are you handling the Data Contracts between these layers now? We found that without strict schema enforcement at the Bronze-to-Silver transition, our AI agents start hallucinating because of upstream schema drift.
I’ve found that the Medallion approach is a lifesaver for data observability; if a downstream Gold dashboard shows a discrepancy, you can perform a root-cause analysis by tracing the logic back to the exact Silver cleansing step or the raw Bronze file to see where the data corrupted.
I completely agree with David. This built-in lineage is the unsung hero of the Medallion pattern. In my current project, we automated this further by using Data Quality checks between the Bronze and Silver layers. It saves our engineering team hours of "data archeology" whenever a pipeline breaks during a high-volume sprint, ensuring our Gold-tier tables remain a "Single Source of Truth" for the executive leadership team.
Ryan, we actually "shifted left" on our governance. We now use a Schema Registry that blocks the ingestion at the source if the contract is violated. This means the Bronze layer stays clean. To answer your point, our dbt models now include "Contract Tests" as part of the CI/CD pipeline. If a dev adds a column to a source without updating the contract, the build fails before it even touches the Silver layer. It’s annoying for devs, but it’s the only way to keep our Gold data "AI-Ready."