Standard Deviation in clojure

Question

I am trying to write a function in clojure to find the standard deviation of a sequence (vector). So far I have defined a function to find the average of a set of numbers, but I am having an issue with a couple of things.

First I am confused over how to use a square root and powers in clojure. Second I am trying to figure out how to pull out each element individually out the vector and subtract the mean from it and then square it.

So far this is my function

(defn mean [a] (/ (reduce + a) (count a))) 

(defn standarddev [a] (Math/sqrt (/ (reduce + (map square #(- % (mean a) a))) (- (count a) 1 ))))

Answer 1

As long as you have a double, you can use Java's Math class (https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html) to perform calculations like sqrt. You don't need to do anything special to access the Math class, because Clojure make all java.lang classes available to you w/o import.

Answer 2

You are pretty close. Assuming you already have the following functions

(defn square [n] (* n n))

(defn mean [a] (/ (reduce + a) (count a)))

2 problems with your standarddev function

(defn standarddev [a] (Math/sqrt (/ (map square (map - a (mean a))) (- (count a) 1 ))))

1) (map - a (mean a)) Doesn't work because you are subtracting a "number" from a "vector". To fix repeat (mean a) as many times as there are elements in "a" Easiest and by no means efficient solution would be

(map - a (repeat (mean a)))

2) (map square (map - a (mean a))) Doesn't work because of #1 above and because map returns a "vector". To fix sum the elements of the vector

(reduce + (map square (map - a (repeat (mean a)))))

Your standard dev function should now be

  (defn standarddev [a] 
      (Math/sqrt (/ 
                    (reduce + (map square (map - a (repeat (mean a))))) 
                    (- (count a) 1 ))))

Answer 3

You can gently increase performance by getting rid of the map altogether

(def square #(* % %))

(defn standard-deviation
  [a]
  (let [mn (mean a)]
    (Math/sqrt
      (/ (reduce #(+ %1 (square (- %2 mn))) 0 a)
         (dec (count a))))))

Answer 4

First I am confused over how to use a square root and powers in clojure.

To square something, just multiply it by itself:

(defn square [n]
  (* n n))

If you want a power higher than 2, you could also use an exponentiation function:

(defn exp [x n]
  (reduce * (repeat n x)))

Second I am trying to figure out how to pull out each element individually out the vector and subtract the mean from it and then square it.

The Clojure (functional) way of iterating through a seq is to use map. Map takes a function and a collection and returns the result of applying that function to each element of the collection.

(defn squares [avg coll] (map #(square (- % avg)) coll))

Final standard-deviation function, using the above 2 functions and your mean:

(defn standard-deviation [coll]
  (let [avg (mean coll)
        squares (squares avg coll)
        total (count coll)]
    (Math/sqrt (/ (reduce + squares) (- total 1)))))

inspiration from: https://github.com/clojure-cookbook/clojure-cookbook/blob/master/01_primitive-data/1-20_simple-statistics.asciidoc

Answer 5

Corrected sample standard deviation, same as in R sd function

(defn sd [abc] 
    (Math/sqrt
        (/ (reduce + (map #(* % %)
            (map #(- % (/ (reduce + abc) (count abc))) abc)))
           (dec (count abc))
        )
    )
)