How to avoid transitive AOT compiling of clojure source files

4
votes

I have a cross Java-clojure project that uses maven as the build tool. AOT compiling of some of the namespaces are required.

The problem is, the compilation takes like forever to complete. Inspecting the maven process gives the following observations:

  1. When compiling, the CPU wait time is very high (single core 70-90%).

  2. Sampling that high wait thread gives a stack where the innermost calls are:

    at java.io.FileDescriptor.sync(Native Method)
at clojure.lang.Compiler.writeClassFile(Compiler.java:7269)
  3. Checking the target/classes/ folder, a great number of .class files are generated, mostly for the clojure-based libraries on which my project depends.

What I have tried:

  1. Instructed the clojure-maven-plugin to compile only given namespaces. But the propagation along the dependency line is not inhibited.
  2. Explicitly included given namespaces, and excluded the namespace of dependency. The dependency library is still recompiled by the AOT process.

Well, just before I pressed the "POST" button, I found this issue. It seems back in 5 years we've had such discussion, and the problem is not specific to clojure-maven-plugin. So what's new about this issue? It takes about 4 minutes to mvn clean compile once on my machine for my not-so-huge project, and I can literally grab a coffee, sit back and relax..

If the issue is not to be resolved in any way in the near future, what might be suggested to mitigate my pain? Any thought are welcome.

1
clojure
what about doing just "maven install"? – DanLebrero
@dAni The caveat is in the clean part. As long as the compiled class files are cleaned, the compilation will start all over again. Though incremental compilation is possible, we do need to clean it from time to time.. – Davyzhu
Clojure 1.8 came out today with dev.clojure.org/jira/browse/CLJ-703. I see 33% faster compilation time in some of my projects. – DanLebrero
@dAni Excellent! This DOES look like a critical performance boost. I'll give it a try and report back the soonest possible. – Davyzhu
After opting to 1.8, my mvn clean compile now took only 40+ seconds! From 4 MINUTES! If using my clean.sh, only 20+ seconds. – Davyzhu

1 Answers

0
votes

As it turns out, there might not be a good solution to this problem yet. I now however resorts to this walk-around:

Instead of running the clean goal, I now use my own clean.sh script to clean only class files in my own namespaces. By doing this, my 4-minute compilation time drops to a little bit more than 1 minute.

I'll leave the question open and perhaps some day we'll see a better solution to it.