I am setting up a distributed Jenkins Pipeline where my initial build runs inside a Docker container, but the deployment needs to happen on a separate physical worker node. I need to move the compiled artifacts from the first stage to the second. What is the correct syntax for using stash and unstash to preserve these files across different workspaces, and are there any specific limitations regarding file size or persistence I should be aware of?
3 answers
The stash command is designed specifically for saving files from the current workspace to the Jenkins controller so they can be retrieved later in the same pipeline run using unstash. For example, in your build stage, you would use stash name: 'my-artifacts', includes: 'target/*.jar'. Then, in your deployment stage on a different node, simply call unstash 'my-artifacts'. It is important to remember that stashes are temporary and are deleted once the pipeline completes. Unlike the archiveArtifacts command, stashing is meant for internal pipeline flow rather than long-term storage. However, avoid stashing very large files (over 100MB) frequently, as this can put significant pressure on the network and storage of your Jenkins controller.
Could you clarify if you need these files to persist after the build is finished, or are they only necessary for the immediate execution of the downstream stages?
Using stash is great because it automatically handles the transfer between different nodes for you. Just make sure your "includes" pattern correctly matches the file structure in your build container.
I agree with Jessica; the glob syntax can be tricky. I always recommend using a very specific "includes" pattern to avoid accidentally stashing thousands of small log files, which can really slow down the unstash process on the target node.
Brian's question is vital because it determines your storage strategy. If you need the files for audit or later use, you should use archiveArtifacts instead. However, for moving data between a build container and a worker node in a single run, stash is much faster. In complex Cloud Technology setups, I often see people use a shared filesystem like NFS or an S3 bucket for massive datasets, but for standard binaries and test reports, the built-in stash mechanism is the cleanest way to maintain a stateless environment across your Jenkins nodes.