Our Spark jobs are crawling because we have millions of 10KB files being ingested from our real-time streaming API. We know this "Small File Problem" is killing our IOPS. What are the best ways to compact these files automatically without taking our data lake offline or causing data consistency issues?
Our experts are ready to help you with any questions about courses, admissions, or career paths. Get personalized guidance from industry professionals.
Request a Call Back
"PMI®", "PMBOK®", "PMP®", "CAPM®" and "PMI-ACP®" are registered marks of the Project Management Institute, Inc. | "CSM", "CST" are Registered Trade Marks of The Scrum Alliance, USA. | COBIT® is a trademark of ISACA® registered in the United States and other countries.
