OpenMP compiler (add-on?) for Common Lisp?

2
votes

Are there any projects dealing with parallelism in Lisp land? I'm reading now the OpenMP's white papers and I'm convinced that this (or something very similar) must be the future of just any multi-purpose programming language.

I've searched the web for mentions of Common Lisp in relation to OpenMP but didn't find much. So, do you know of any project that would use Common Lisp in connection with data and task parallelism?

Just to give some sort of illustration of a feature I would imagine such library would provide:

(defprocess some-process
  (<list of processes to inherit transactions from>)
  (<list of transaction defs>)
  (<process creation arguments>)
  "<docstring>"
  (accept
   (some-transaction (<transaction arglist>) ...)
   (some-other-transaction (...) ...))
  ...
  (select ...))

I.e. it would be a framework for managing processes, which builds on top of multithreading.

I've researched both libraries mentioned by Lars Brinkhoff and here's very quick overview:

  • lparallel is a library based on another Lisp threading library, bordeaux-threads. It also implements some common functions based on threads (parallel reduce, map etc) as well as couple of macros (parallel let). It also has few new constructs such as parallel tree (a construct managing parallel executions + barriers of hierarchy of promises - basic building blocks of parallel semantics). The benefits, as I see them:
    • easier to use,
    • more portable,
    • incurs no penalty on the programmer who wants to exploit typical for Lisp long math, as well as Lisp-specific coding primitives, like for example, error handling.
  • mpi bindings. mpi by itself is a very low-level library. It builds upon semantics and limitations of C and Fortran languages making it difficult to adapt to Lisp. However, it probably performs better (I didn't run any tests yet), it is more low-level, and it comes with the framework for executing the code on clusters / networks of computers. It would be painstakingly difficult to pass around Lisp objects using this library. Even integers would present a major challenge because the library is very down-to-metal and deals with machine words rather then numbers. There are also problems running and debugging programs because mpi would start several instances of Lisp competing with each other for input and output...

I will try to set up a local network on my two machines and see if I can come up with a reasonable setup for this kind of parallel computing. Until then it looks like using lparallel is by far the easiest way to go about parallel programming in Lisp.

1
parallel-processingcommon-lispopenmp
See stackoverflow.com/a/9400913/1193075 – uselpa
Thanks @uselpa yup, OpenMP is something like Concurrent C mentioned in that answer, as far as I can tell. But the idea of OpenMP isn't only to provide some means of parallelism, but also to modify existing language constructs to work in parallel. So, for example, in addition to loop macro there would be some kind of openmp:loop macro that would expand into parallel code. Well, this is the idea more or less. – user797257

1 Answers