Cannot clone the same repository more than twice simultaniously with 3 Jenkins parallel steps? index-pack failed?

0
votes
stage(code_scan, ut & build) {    
     parallel static_code_scan: { 
     }, unit_test: {
     }, build: {
     }
}

I have a Pull Request Pipeline with a build stage that runs a shell script that starts a static code scan, then unit test and finally build stage. I have split the script using 3 case statements, each called by 3 parallel jobs in my pipeline. This Job runs on either 2 Build nodes, I believe the way I have my parallel script setup it cannot handle more then 2 parallel jobs at the same time, the issue is with index-pack.

  • Should I instead have a separate stage for static code scan, which only takes about 5 minutes, and one stage for unit and build which take longer, instead of one stage with 3 parallel stages?

The issue is that all 3 steps need to git clone a repository to run each task. I am able to do so with only 2 parallel steps but not all 3 together. This is the error I get:

17:09:00 + git clone [email protected]:[REPO].git -b master .
17:09:00 Cloning into '.'...
17:09:02 fatal: Unable to read current working directory: No such file or directory
17:09:02 fatal: index-pack failed
2
linuxbashjenkinsjenkins-pipelinejenkins-groovy

2 Answers

0
votes

you can try it like this, it will go smooth

node {
    deleteDir()
    parallel build: {
        dir('build') {
            git url: "https://github.com/dontknow/test0.git", branch: "main"
            sh 'ls -al'
        }        
    }, scan: {
        dir('scan') {
            git url: "https://github.com/dontknow/test0.git", branch: "main"
            sh 'ls -al'
        } 
    }, test : {
        dir('test') {
            git url: "https://github.com/dontknow/test0.git", branch: "main"
            sh 'ls -al'
        } 
    }
}
0
votes

If three parallel stages run on two nodes, you may hit a case where two stages run on the same node and in the same directory. Trying to clone a repo in the same directory twice can fail. You can try a dir() directive to force a differently-named directory, or provide an agent for each stage, which will guarantee a unique directory for a stage.