DSL for Clojure image synthesis

6
votes

I'm experimenting with creating a small library/DSL for image synthesis in Clojure. Basically the idea is to allow users of the library to compose sets of mathematical functions to procedurally create interesting images.

The functions need to operate on double values, and take the form of converting a location vector into a colour value, e.g. (x,y,z) - > (r,g,b,a)

However I'm facing a few interesting design decisions:

  • Inputs could have 1,2,3 or maybe even 4 dimensions (x,y,z plus time)
  • It would be good to provide vector maths operations (dot products, addition, multiplication etc.)
  • It would be valuable to compose functions with operations such as rotate, scale etc.
  • For performance reasons, it is important to use primitive double maths throughout (i.e. avoid creating boxed doubles in particular). So a function which needs to return red, green and blue components perhaps needs to become three separate functions which return the primitive red, green and blue values respectively.

Any ideas on how this kind of DSL can reasonably be achieved in Clojure (1.4 beta)?

2
image-processingclojuredsl
Got any early draft pieces of it up on github? - rplevy
github.com/mikera/clisk - mikera
are you aware of pan (and my pancito, which was a pure-haskell copy). that was a haskell-based library that did just this. you can probably google up some info. it might give you some ideas. [...] conal.net/Pan (if you're hoping to generate something beautiful, don't be discouraged - i am still amazed how conal managed to generate such ugly images from such a powerful library). ps pancito too acooke.org/pancito-code - andrew cooke

2 Answers

1
votes

A look at the awesome ImageMagick tools http://www.imagemagick.org can give you an idea of what kind of operations would be expected from such a library.

Maybe you'll see that you won't need to drop down to vector math if you replicate the default IM toolset.

0
votes

OK, so I eventually figured out a nice way of doing this.

The trick was to represent functions as a vector of code (in the "code is data" sense, e.g. 

[(Math/sin (* 10 x)) 
 (Math/cos (* 12 y)) 
 (Math/cos (+ (* 5 x) (* 8 y)))]

This can then be "compiled" to create 3 objects that implement a Java interface with the following method:

public double calc(double x, double y, double z, double t) {
    .....
}

And these function objects can be called with primitive values to get the Red, Green and Blue colour values for each pixel. Results are something like:

enter image description here

Finally, it's possible to compose the functions using a simple DSL, e.g. to scale up a texture you can do:

(vscale 10 some-function-vector)

I've published all the code on GitHub for anyone interested:

https://github.com/mikera/clisk