18
votes

I was reading about clojure reducers introduced in 1.5, here: https://github.com/clojure/clojure/blob/master/changes.md. My understanding is that they're a performance enhancement on the existing map/filter/reduce function(s). So if that's the case, I'm wondering why they are in a new namespace, and do not simply replace the existing map/reduce/filter implementations. Stated differently, why would I ever not choose to use the new reducers feature?

EDIT:

In response to the inital two answers, here is a clarification:

I'm going to quote the release notes here:

Reducers provide a set of high performance functions for working with collections. The actual fold/reduce algorithms are specified via the collection being reduced. This allows each collection to define the most efficient way to reduce its contents.

This does not sound to me like the new map/filter/reduce functions are inherently parallel. For example, further down in the release notes it states:

It contains a new function, fold, which is a parallel reduce+combine

So unless the release note are poorly written, it would appear to me that there is one new function, fold, which is parallel, and the other functions are collection specific implementations that aim to produce the highest performance possible for the particular collection. Am I simply mis-reading the release notes here?

4
Good question although "[W]hy would I ever not choose to use the new [clojure] reducers feature" serves as a better title.mike3996
thank you for the feedback, done.Kevin
This might help understanding why reducers cannot just replace the old implementations: vimeo.com/45561411Joe Lehmann

4 Answers

9
votes

Foreword: you have problem and you are going to use parallelism, now problems two have you.

They're replacement in a sense they do that work in parallel (versus plain old sequential map and etc). Not all operations could be parallelized (in many cases operation has to be at least associative, also think about lazy sequences and iterators). Moreover, not every operation could be parallelized efficiently (there is always some coordination overhead, sometimes overhead is greater than parallelization gain).

8
votes

They cannot replace the old implementations in some cases. For instance if you have infinite sequences or if you actually require sequential processing of the collection.

1
votes

A couple of good reasons you might decide not to use reducers:

  • You need to maintain backwards compatibility with Clojure 1.4. This makes it tricky to use reducers in library code, for example, where you don't know what Clojure version your uses will be using
  • In some circumstances there are better options: for example if you are dealing with numerical arrays then you will almost certainly be better off using something like core.matrix instead.
0
votes

I found the following write up by Rich Hickey that while still somewhat confusing, cleared (some) things up for me: http://clojure.com/blog/2012/05/08/reducers-a-library-and-model-for-collection-processing.html

In particular the summary:

By adopting an alternative view of collections as reducible, rather than seqable things, we can get a complementary set of fundamental operations that tradeoff laziness for parallelism, while retaining the same high-level, functional programming model. Because the two models retain the same shape, we can easily choose whichever is appropriate for the task at hand.