Jenkins 2.176.4-3 rolling
Gradle 4.3.1
Issue area: Parallel run of a given single Gradle task (or it could be any simple action) and especially when running concurrent runs of Jenkinsfile based pipelines
All the sudden I got this error in Jenkins log page, never seen this error before (found no stackoverflow posts either for this error in Jenkins).
Error: java.lang.IllegalArgumentException: Last unit does not have enough valid bits
For some reason the previous build failed and automatically took me to Jenkins logs page showing Jenkins angry fire face:
Re-ran just one run of Jenkinsfile pipeline again (which runs a single Gradle task (lets say:
rpmtask) present in multiple projects in parallel) and it worked successfully!!; checked parameters/values passed between successful and failed runs - they were similar.
Example Jenkinsfile can be found here: Jenkinsfile Pipeline DSL: How to Show Multi-Columns in Jobs dashboard GUI - For all Dynamically created stages - When within PIPELINE section where I'm running a gradle task (lets say gradle rpm) in parallel (each gradle rpm task running in individual stage step) by creating dynamic stages (so that I don't hard-code them in Jenkinsfile for each project).
NOTE: In Jenkinsfile, even though the gradle task rpm (that I'm running in parallel in individual stage (under stages) section has a valid gradle task defined i.e. rpm, it seems like Gradle task running on multiple projects in parallel (section within Jenkinsfile) is not 100% successful everytime, with at least Gradle 4.3.1 as I'm also seeing this other error:
05:31:41 FAILURE: Build failed with an exception.
05:31:42
05:31:42 * What went wrong:
05:31:42 Task 'null' not found in root project 'T000123_ABCD_Project'.
05:31:42
This issue poster said similar here: gradle issue after distribution 4.10.2
Here is the Full error log in Jenkins stack trace is showing:
Oops!
A problem occurred while processing the request.
Please check our bug tracker to see if a similar problem has already been reported.
If it is already reported, please vote and put a comment on it to let us gauge the impact of the problem.
If you think this is a new issue, please file a new issue.
When you file an issue, make sure to add the entire stack trace, along with the version of Jenkins and relevant plugins.
The users list might be also useful in understanding what has happened.
Stack trace
java.lang.IllegalArgumentException: Last unit does not have enough valid bits
at java.util.Base64$Decoder.decode0(Base64.java:734)
at java.util.Base64$Decoder.decode(Base64.java:526)
at hudson.util.UnbufferedBase64InputStream.read(UnbufferedBase64InputStream.java:41)
at hudson.util.UnbufferedBase64InputStream.read(UnbufferedBase64InputStream.java:53)
at java.io.DataInputStream.readFully(DataInputStream.java:195)
at java.io.DataInputStream.readFully(DataInputStream.java:169)
at hudson.console.ConsoleNote.readFrom(ConsoleNote.java:258)
at hudson.console.ConsoleAnnotationOutputStream.eol(ConsoleAnnotationOutputStream.java:111)
at hudson.console.LineTransformationOutputStream.eol(LineTransformationOutputStream.java:60)
at hudson.console.LineTransformationOutputStream.write(LineTransformationOutputStream.java:56)
at java.io.FilterOutputStream.write(FilterOutputStream.java:77)
at org.jenkinsci.plugins.workflow.log.FileLogStorage$1$1.write(FileLogStorage.java:238)
at java.io.FilterOutputStream.write(FilterOutputStream.java:125)
at org.apache.commons.io.output.ProxyOutputStream.write(ProxyOutputStream.java:89)
at org.kohsuke.stapler.framework.io.LargeText$HeadMark.moveTo(LargeText.java:314)
at org.kohsuke.stapler.framework.io.LargeText.writeLogTo(LargeText.java:229)
at hudson.console.AnnotatedLargeText.writeRawLogTo(AnnotatedLargeText.java:168)
at org.jenkinsci.plugins.workflow.log.FileLogStorage$1.writeHtmlTo(FileLogStorage.java:203)
at hudson.console.AnnotatedLargeText.writeLogTo(AnnotatedLargeText.java:147)
at org.kohsuke.stapler.framework.io.LargeText.doProgressText(LargeText.java:264)
at hudson.console.AnnotatedLargeText.doProgressiveHtml(AnnotatedLargeText.java:95)
at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:535)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
Caused: javax.servlet.ServletException
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:797)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
at org.kohsuke.stapler.MetaClass$2.doDispatch(MetaClass.java:219)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
at org.kohsuke.stapler.MetaClass$9.dispatch(MetaClass.java:456)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:280)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:280)
at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
at org.kohsuke.stapler.Stapler.invoke(Stapler.java:676)
at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:873)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
at com.cloudbees.jenkins.support.impl.cloudbees.UnrestrictedApiCallsMonitor$ApiMonitorFilter.doFilter(UnrestrictedApiCallsMonitor.java:120)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at org.jenkinsci.plugins.ssegateway.Endpoint$SSEListenChannelFilter.doFilter(Endpoint.java:243)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at com.cloudbees.jenkins.support.slowrequest.SlowRequestFilter.doFilter(SlowRequestFilter.java:37)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at io.jenkins.blueocean.auth.jwt.impl.JwtAuthenticationFilter.doFilter(JwtAuthenticationFilter.java:61)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at io.jenkins.blueocean.ResourceCacheControl.doFilter(ResourceCacheControl.java:134)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1701)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1668)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:745)
What's making me get this error message in Jenkins?
Error: java.lang.IllegalArgumentException: Last unit does not have enough valid bits
and
Why my pipeline runs fail intermittently with the following error message in Gradle (when parallel section is used to run something (i.e. a Gradle someTask) within Jenkinsfile)?
Task 'null' not found in root project
When running many concurrent runs of this pipeline (which runs a Gradle task on many projects in parallel), let's say I launch 4 pipelines runs and each is building 3-5 project's Gradle rpm task), then I saw the following issues related to FILE locking.
https://stackoverflow.com/a/59828842/1499296
i.e. Gradle not able to delete folder/file (as listed below)
**Execution failed for task ':clean'.
> Unable to delete directory: /some/path/build/someFolder/aFolder_or_File **
or
Failed to capture snapshot of output files for task due to Timeout waiting to lock file hash cache
Failed to capture snapshot of output files for task ':rpm' property 'archivePath' during up-to-date check.
> Timeout waiting to lock file hash cache (/view/user123456_Team_Tools/vobs/space/test/folder1/Project1of5/.gradle/4.3.1/fileHashes). It is currently in use by another Gradle instance.
Owner PID: 29003
Our PID: 903
Owner Operation:
Our operation:
Lock file: /view/user123456_Team_Tools/vobs/space/test/folder1/Project1of5/.gradle/4.3.1/fileHashes/fileHashes.lock
or
getting: Could not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory()
17:40:11 FAILURE: Build failed with an exception.
17:40:11
17:40:11 * What went wrong:
17:40:11 Could not create service of type ScriptPluginFactory using BuildScopeServices.createScriptPluginFactory().
17:40:11 > Could not create service of type FileHasher using BuildSessionScopeServices.createFileSnapshotter().
or
Could not read path '/some/jenkins/workspace/path/of/a/job/folder/sub-folder/file.class_or_any_extension'
or
java.io.FileNotFoundException i.e. Not able to find a file (that Gradle is builds successfully if I run only one run of this pipeline).
17:38:28 * What went wrong:
17:38:28 Execution failed for task ':rpm'.
17:38:28 > java.io.FileNotFoundException: '/some/jenkins/workspace/path/of/a/job/folder/sub-folder/artifact-1.0.0.121.rpm'
