Our Docker builds in CodeBuild are taking over 15 minutes because we are downloading heavy dependencies every time. We are using the standard Ubuntu runtime. Are there caching strategies within 'buildspec.yml' or custom Docker images that can bring this down to under 5 minutes? We're particularly looking for ways to cache Docker layers effectively.
3 answers
To speed up Docker builds, you should enable 'Local Caching' in your CodeBuild project settings. Specifically, select 'Docker Layer Cache'. This persists the layers on the build host so they don't need to be pulled from the registry every time. In your 'buildspec.yml', make sure you are using the '--cache-from' flag in your 'docker build' command. This tells Docker to use existing images as a source for cached layers. We managed to drop our build times from 12 minutes to just 3 minutes by implementing this alongside a custom base image that already contains our heaviest SDKs.
Does the 'Local Caching' feature work if we are using a 'Small' compute instance, or do we need to upgrade to 'Large' to see a significant difference in I/O performance? I’m worried that the disk throughput on the smaller instances might negate the benefits of caching.
You should also check out 'S3 Caching' for your node_modules or Maven dependencies. It’s separate from Docker layer caching but equally important for overall speed.
Spot on, Nancy. Combining S3 caching for app dependencies with Docker layer caching for the environment is the "golden rule" for AWS CodeBuild optimization.
In my experience, Charles, the compute size mostly affects CPU-bound tasks like compilation. For Docker layer caching, the network and disk I/O are the bottlenecks. However, upgrading to a 'Medium' or 'Large' instance does provide better burstable network performance, which helps when you do have to pull a new base image. It's definitely worth the extra few cents per build if it saves your developers 10 minutes of waiting.