loading configuration file in clojure as data structure

Question

Is there a reader function in clojure to parse clojure data structure? My use case is to read configuration properties files and one value for a property should be a list. I'd like to be able to write this as:

file.properties:

property1 = ["value1" "value2"]

and in clojure:

(load-props "file.properties")

and get a map with value {property1, ["value1" "value2"]

Right now,m I'm doing the following, with the same input file "file.properties":

(defn load-props [filename]
    (let [io (java.io.FileInputStream. filename)
        prop (java.util.Properties.)]
    (.load prop io)
    (into {} prop)))

;; returns:
;; {"property1" "[\"valu1\", \"valu2\"]"}
(load-props "file.properties")

But I cannot get a way to parse the result to a clojure's vector. I'm basically looking for something like Erlang's file:consult/1 function. Any idea how to do this?

Answer 1

java.util.Properties implements Map so this can be done very easily without manually parsing properties files:

(require 'clojure.java.io)
(defn load-props
  [file-name]
  (with-open [^java.io.Reader reader (clojure.java.io/reader file-name)] 
    (let [props (java.util.Properties.)]
      (.load props reader)
      (into {} (for [[k v] props] [(keyword k) (read-string v)])))))

(load-props "test.properties")
;=> {:property3 {:foo 100, :bar :test}, :property2 99.9, :property1 ["foo" "bar"]}

In particular, properties files are more complicated than you think (comments, escaping, etc, etc) and java.util.Properties is very good at loading them.

Answer 2

If you want to read java-style properties files, look at Dave Ray's answer - though properties files have many limitations.

If you are using Clojure 1.5 or later, I suggest you use edn, the extensible data notation used in Datomic - it's basically clojure data structures, with no arbitrary code execution, and the ability to add tags for things like instances or arbitrary types.

The simplest way to use it is via read-string and slurp:

(require 'clojure.edn)
(clojure.edn/read-string (slurp "filename.edn"))

That's it. Note that read-string only reads a single variable, so you should set up your configuration as a map:

{ :property1 ["value1" "value2"] }

Then:

(require 'clojure.edn)
(def config (clojure.edn/read-string (slurp "config.edn")))
(println (:property1 config))

returns

["value1" "value2"]

Answer 3

Is there a reader function in clojure to parse clojure data structure?

Yes. It's called read. You can also use it to read configuration data.

A file props.clj containing

{:property1 ["value1" 2]
 :property2 {:some "key"}}

can be read like this:

(ns somens.core
  (:require [clojure.java.io :as io])
  (:import [java.io PushbackReader]))

(def conf (with-open [r (io/reader "props.clj")]
            (read (PushbackReader. r))))

When reading untrusted sources it might be a good idea to turn of *read-eval*:

(def conf (binding [*read-eval* false]
            (with-open [r (io/reader "props.clj")]
              (read (PushbackReader. r)))))

For writing configuration data back to a file you should look at print functions such as pr and friends.

Answer 4

contrib has functions for reading writing properties,

http://richhickey.github.com/clojure-contrib/java-utils-api.html#clojure.contrib.java-utils/as-properties

If this is for your own consumption then I would suggest reading/writing clojure data structures you can just print them to disk and read them.

Answer 5

(use '[clojure.contrib.duck-streams :only (read-lines)])
(import '(java.io StringReader PushbackReader))

(defn propline->map [line] ;;property1 = ["value1" "value2"] -> { :property1  ["value1" "value2"] }
  (let [[key-str value-str] (seq (.split line "="))
        key (keyword (.trim key-str))
        value (read (PushbackReader. (StringReader. value-str)))]
        { key value } ))

(defn load-props [filename]
  (reduce into (map propline->map (read-lines filename))))

DEMO

user=> (def prop (load-props "file.properties"))
#'user/prop
user=> (prop :property1)
["value1" "value2"]
user=> ((prop :property1) 1)
"value2"

UPDATE

(defn non-blank?   [line] (if (re-find #"\S" line) true false))
(defn non-comment? [line] (if (re-find #"^\s*\#" line) false true))

(defn load-props [filename]
  (reduce into (map propline->map (filter #(and (non-blank? %)(non-comment? %)) (read-lines filename)))))