I've been reading about the "Zero Copy" revolution with Data Cloud. As a Technical Architect, I'm trying to understand if this truly eliminates the need for traditional ETL tools and middleware like MuleSoft when connecting to external data lakes like Snowflake or AWS S3. How does this impact system latency?
3 answers
Zero Copy is a paradigm shift because it allows Salesforce to "read" data directly from your external data lake without physically moving or duplicating it. This significantly reduces storage costs and ensures that you are always working with the "source of truth." In terms of latency, since the data isn't being synced periodically, you avoid the lag associated with batch jobs. However, it's not a total replacement for middleware. You still need tools like MuleSoft for complex transformations and orchestration between systems that don't support the Zero Copy protocol. For high-volume read operations, it’s much faster, but you must ensure your external federated database is optimized for the queries Salesforce will be sending its way via the metadata layer.
Do you think this will eventually make the standard Salesforce "Big Objects" obsolete for long-term data archiving?
It definitely simplifies the security model since you aren't managing multiple copies of sensitive PII across different platforms.
Exactly, Jennifer. Keeping the data at the source makes GDPR and CCPA compliance much easier to manage globally.
That’s a valid question, Michael. While Zero Copy is great for active data, Big Objects still serve a purpose for internal compliance where data must reside within the Salesforce infrastructure. However, for most analytical use cases, federating that data to an external lake and using Zero Copy is definitely the more modern and scalable path forward for enterprise architectures. POSTED BY: David Anderson DATE: 21-05-2024Comment: That’s a valid question, Michael. While Zero Copy is great for active data, Big Objects still serve a purpose for internal compliance where data must reside within the Salesforce infrastructure. However, for most analytical use cases, federating that data to an external lake and using Zero Copy is definitely the more modern and scalable path forward for enterprise architectures.