Sonarqube 6.7 compute engine errors

Question

Anyone have any clue about this?

"Directory has not been set yet" - Don't know what that means, other projects work fine, This one is rather large looking at 13,500 java files, another 2500 other files. Analysis on build server finished fine, but the CE on the web server is having an issue with this one project.

Smaller Projects are no issue.

Also, anyone know how I can re-run this task on the server without doing a whole hour of sonar-runner again?

2017.11.22 09:43:03 INFO  ce[][o.s.ce.app.CeServer] Compute Engine is operational
2017.11.22 10:17:07 INFO  ce[AV_kFoAKPGI1NQcqJopo][o.s.c.t.CeWorkerImpl] Execute task | project=large_java_project | type=REPORT | id=AV_kFoAKPGI1NQcqJopo | submitter=gbizeau
2017.11.22 10:17:07 ERROR ce[AV_kFoAKPGI1NQcqJopo][o.s.s.c.t.s.ComputationStepExecutor] Execution of listener failed
java.lang.IllegalStateException: Directory has not been set yet
        at org.sonar.server.computation.task.projectanalysis.batch.BatchReportDirectoryHolderImpl.getDirectory(BatchReportDirectoryHolderImpl.java:37)
        at org.sonar.server.computation.task.projectanalysis.batch.BatchReportReaderImpl.ensureInitialized(BatchReportReaderImpl.java:53)
        at org.sonar.server.computation.task.projectanalysis.batch.BatchReportReaderImpl.readContextProperties(BatchReportReaderImpl.java:222)
        at org.sonar.server.computation.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.createProjectAnalysis(PostProjectAnalysisTasksExecutor.java:123)
        at org.sonar.server.computation.task.projectanalysis.api.posttask.PostProjectAnalysisTasksExecutor.finished(PostProjectAnalysisTasksExecutor.java:103)
        at org.sonar.server.computation.task.step.ComputationStepExecutor.executeListener(ComputationStepExecutor.java:71)
        at org.sonar.server.computation.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:56)
        at org.sonar.server.computation.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:73)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:134)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:97)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:81)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:73)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:43)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
2017.11.22 10:17:07 ERROR ce[AV_kFoAKPGI1NQcqJopo][o.s.c.t.CeWorkerImpl] Failed to execute task AV_kFoAKPGI1NQcqJopo

Additional Error...

java.lang.IllegalStateException: Fail to select data of CE task AV_kFoAKPGI1NQcqJopo
        at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:74)
        at org.sonar.server.computation.task.projectanalysis.step.ExtractReportStep.execute(ExtractReportStep.java:59)
        at org.sonar.server.computation.task.step.ComputationStepExecutor.executeSteps(ComputationStepExecutor.java:64)
        at org.sonar.server.computation.task.step.ComputationStepExecutor.execute(ComputationStepExecutor.java:52)
        at org.sonar.server.computation.task.projectanalysis.taskprocessor.ReportTaskProcessor.process(ReportTaskProcessor.java:73)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.executeTask(CeWorkerImpl.java:134)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.findAndProcessTask(CeWorkerImpl.java:97)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.withCustomizedThreadName(CeWorkerImpl.java:81)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:73)
        at org.sonar.ce.taskprocessor.CeWorkerImpl.call(CeWorkerImpl.java:43)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: org.postgresql.util.PSQLException: ERROR: invalid memory alloc request size 1315662807
        at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2476)
        at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2189)
        at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:300)
        at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:428)
        at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:354)
        at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:169)
        at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:117)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
        at org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:96)
        at org.sonar.db.ce.CeTaskInputDao.selectData(CeTaskInputDao.java:67)
        ... 17 common frames omitted

UPDATE: So I have been unable thus far to "tune" postgresql to be able to handle these larger sizes. Our project was 3GB uncompressed. I found 2GB of static code that did not need to be in our build folder (rpms and such) and removed them, SonarQube now functions. Although this is not a fix to the problem, I am hoping that others can use this info running into the same problem.

What OS do you use? Maybe you have a directory path length problem. — Jeroen Heier
Linux Centos 7, The scan is rather large, It's 637MB Compressed... I wonder if it's can't extract it or it's not done extracting before it tries to start? — Glen Bizeau
Maybe you can find something during a code scan of the SonarQube source code — Jeroen Heier
If you check ce.log, do you see any other exception? The error was caused by the fact that at a earlier stage of the processing of the report, it failed to set the path of the unzipped directory internally. — Duarte Meneses
I updated with all the data that I can see in relation to this scan failure. Anything else in the logs before that was successful scans. — Glen Bizeau

Seb - SonarSource Team Seb - SonarSource Team · Accepted Answer · 2017-11-23T09:23:40

I'm afraid the stacktrace you are reporting is hiding another error, which would be the real issue.

The reported error occurs while building arguments to invoke the public PostTask API in the Compute Engine. You can look at this code as if executed in a finally block. I've opened a ticket to fix this hiding problem SONAR-10115.

However, the error does indirectly confirm that there is a problem with extracting the report. We "just" don't know which one exactly. We can assume it's related to the size of the report.

The ZIP of the report is fetched from the Database and streamly extracted to disk.

Here are some ideas of what could have gone wrong:

your database is not able to stream the whole file (timeout or whatever) to the server
extracting to disk fails because you're running out of free space (depends on DB vendor and configuration, network, ...)
or reaching a limit in the number of files and/or directories (depends on OS)
streaming is not working as expected and an OOM was raised

Sonarqube 6.7 compute engine errors

1 Answers