Why does D compilation take so long?

10
votes

D is one of the fastest programming languages to compile, if not the fastest, but this isn't always the case. Things become painfully slow when unittest is turned on. My current project has 6-7 modules (~2000 LOC), with every single one of them having unittests that also contain benchmarks. Here are some numbers from my current project:

dmd -O -noboundscheck takes 0m1.287s

dmd -O -release -noboundscheck takes 0m1.382s

dmd -O -inline -noboundscheck takes 0m1.499s

dmd -O -inline -release -noboundscheck takes 0m3.477s

adding -unittest to any one of the above will drastically increase compilation time:

dmd -O -inline -release -noboundscheck -unittest takes 0m21.918s

and sometimes it crashes DMD:

time dmd -O t1.d -inline -noboundscheck -version=Double -unittest takes 0m2.297s Internal error: ../ztc/gdag.c 776

Evidentially, unittest is buggy but at the same time it has become an important part of my project. I would like to know if the slowdown is normal or is it something that's being worked on? My project is growing and with every new unittest the compilation is taking longer and longer. The only solution I know is to disable -release and -inline, but that's not always desirable.

3
compilationd
Do you have many template instantiations in your unit tests? - Vladimir Panteleev
@CyberShadow Yes, it's mostly template instantiations because there is a lot generic programming. - Arlen
That's most likely the reason. Try moving the non-generic parts of your code outside the templates. - Vladimir Panteleev
@CyberShadow there is no non-generic code. - Arlen
@Arlen, I think he means try reducing the amount of generic code by only making the parts that are truly generic, generic. Often you can mitigate the problem by factoring out pieces of code from your templates that don't really have to be templated. It can also help even if you factor out such code into smaller templates that can be reused more often. - jA_cOp

3 Answers

4
votes

DMD has a known issue with optimisations: long blocks of code optimise with an O(n^2) algorithm, so long functions take a long time to compile with optimisations.

Try splitting your code up into smaller functions and you should get better compile times in the meantime. You can do this quite easily by using inline functions:

void foo()
{
    // lots of code
    // more code
}

Turn this into:

void foo()
{
    void block1()
    {
        // lots of code
    }
    block1();

    void block2()
    {
        // more code
    }
    block2();
}

This worked for me.

1
votes

A very tiny performance improvement could be to move template instantiation to module-scope, via a version(unittest) block, e.g.:

auto foo(T)(T t) { return t; }

version(unittest) {
    alias foo!int fooInt;
}

unittest {
    auto x = fooInt(1);
}

Profiling this, I get around ~30msec speed improvement if I use the aliased template instance in 5000 equivalent unittest blocks via auto x = fooInt(1), compared to instantiating it directly in each unittest block via auto x = foo(1) (this actually expands to auto x = foo!int(1)).

This will likely only work for cases where you have a lot of unittests that create the same template instance.

0
votes

I did replace much of my generic code, but it only reduced compilation time by 4-5 seconds. Things have gotten worse, and I believe the compiler is probably the issue:

time dmd -O -inline -release -noboundscheck -unittest takes 0m30.388s

time dmd -O -inline -release -noboundscheck takes 0m11.597s

time dmd -inline -release -noboundscheck -unittest takes 0m1.884s

When -O, -inline, -release, and -unittest are all set, compilation takes the longest. Dropping -O drastically reduces compilation time. Therefore, to reduce compilation time while unittesting, drop the optimization flag(s). For normal compilations, you could use either of the three (-inline, -release, -unittest) with no problem. In my experience, it's the combination of all three that causes compilation to take the second longest, and the longest when -unittest is set as well.