Is Elixir slower than Ruby?

Question

Please help me solve a benchmark question about Elixir vs. Ruby performance.

I tried to implement the same factorial in both languages, and Ruby shows better results than Elixir:

# ruby_factorial_with_iterator.rb
def factorial_with_iterator(n)
  res = 1
  (1..n).each{|time| res *= time}
  res
end

p "factorial_with_iterator(200000)"
p factorial_with_iterator(200000)

After run:

$ time ruby ruby_factorial_with_iterator.rb
real  0m18.378s
user  0m17.348s
sys   0m0.844s

and two Elixir examples:

# elixir_factorial_with_iterator.exs
defmodule FactorialWithIterator do
  def of(n) do
    Enum.reduce(1..n, 1, &*/2)
  end
end

IO.puts "Factorial of 200000: "
IO.puts FactorialWithIterator.of(200000)

After run:

$ time elixir elixir_factorial_with_iterator.exs
real  1m1.735s
user  1m1.556s
sys   0m0.104s

Another example:

# elixir_factorial_with_recursion.exs
defmodule FactorialWithRecursion do
  def of(0), do: 1
  def of(n) when n > 0 do
    n * of(n - 1)
  end
end

IO.puts "Factorial of 200000: "
IO.puts FactorialWithRecursion.of(200000)

After run:

$ time elixir elixir_factorial_with_recursion.exs
real  1m7.149s
user  1m6.248s
sys   0m0.092s

Why is there such a huge difference: Elixir - 1m1s, and Ruby - just 18s? Or how to write correct iteration code in the Elixir?

P.S. Environment:

• Ubuntu 16.04
• ruby 2.3.1p112 (2016-04-26 revision 54768) [x86_64-linux]
• Erlang/OTP 19 [erts-8.3] [source-d5c06c6] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]
• Elixir 1.4.4

Answer 1

As mentioned in one of the comments, you're using time, which also times the launching of the VM and, in elixir's case, the compiling of the code to BEAM bytecode. To avoid counting all that, you should use benchmarking tools in the languages themselves.

I was curious, so I tried benchmarking these function myself.

I used:

• benchee for elixir: https://github.com/PragTob/benchee
• benchmark-ips for ruby: https://github.com/evanphx/benchmark-ips

Ruby:

require 'benchmark/ips'

def factorial_with_iterator(n)
  res = 1
  (1..n).each{|time| res *= time}
  res
end

Benchmark.ips do |x|
  x.config(time: 5, warmup: 2)
  x.report('factorial_with_iterator.rb') do
    factorial_with_iterator(200000)
  end
  x.compare!
end

Elixir:

defmodule Factorial do
  def iter(n) do
    Enum.reduce(1..n, 1, &*/2)
  end

  def recur(0), do: 1

  def recur(n) when n > 0 do
    n * recur(n - 1)
  end
end

Benchee.run(%{
  "factorial_with_iter.ex" => fn -> Factorial.iter(200000) end,
  "factorial_with_recur.ex" => fn -> Factorial.recur(200000) end
})

I got these results:

Ruby:

Warming up --------------------------------------
factorial_with_iterator.rb
                         1.000  i/100ms
Calculating -------------------------------------
factorial_with_iterator.rb
                          0.033  (± 0.0%) i/s -      1.000  in  29.994713s

Elixir:

Name                              ips        average  deviation         median         99th %
factorial_with_iter.ex         0.0395        25.29 s     ±0.00%        25.29 s        25.29 s
factorial_with_recur.ex        0.0368        27.17 s     ±0.00%        27.17 s        27.17 s

Comparison:
factorial_with_iter.ex         0.0395
factorial_with_recur.ex        0.0368 - 1.07x slower

So these results show Elixir being slightly faster with both implementations with Ruby taking ~30 seconds and Elixir taking ~25 and ~27 seconds.

Using "iterations per second" though, for function which take much longer than a second might be a little "wrong". So I also tried with a much lower input. Instead of 200_000, I used 1_000, and got these results:

Ruby:

Warming up --------------------------------------
factorial_with_iterator.rb
                       169.000  i/100ms
Calculating -------------------------------------
factorial_with_iterator.rb
                          1.750k (± 8.0%) i/s -      8.788k in   5.064619s

Elixir:

Name                              ips        average  deviation         median         99th %
factorial_with_recur.ex        3.15 K      317.36 μs    ±12.72%         306 μs      481.87 μs
factorial_with_iter.ex         3.02 K      331.13 μs    ±16.83%         316 μs         559 μs

Comparison:
factorial_with_recur.ex        3.15 K
factorial_with_iter.ex         3.02 K - 1.04x slower

Curiously, this shows Elixir being much faster than Ruby. Elixir was able to perform over 3k iteration per second for both implementations, where Ruby was only able to perform 1.75k iteration per second.

Using:

• ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin16]
• Elixir 1.6.5 (compiled with OTP 19) running on Erlang/OTP 21 [erts-10.0] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe] [dtrace]