Getting and passing structs by value in Clojure with JNA

Question

I have a C API that I'm trying to use within Clojure, through the JNA API. My issue can best be demonstrated with the following example. Say I have this C code in a library:

typedef struct {
    int foo;
    int bar;
    double baz;
} MyStruct;

MyStruct createStruct() {
    MyStruct myStruct;
    myStruct.foo = 3;
    myStruct.bar = 4;
    myStruct.baz = 3.14;

    return myStruct;
}

double addStruct(MyStruct myStruct) {
    return myStruct.foo + myStruct.bar + myStruct.baz;
}

In this example, I'd like to call createStruct, and then pass that result to addStruct. The important point here is that MyStruct is passed by value as both a return type and an argument. At no point do I need to actually read the values of the fields in MyStruct.

Additionally, in my system, native functions are wrapped like this:

; `quux` is defined in `some-lib` and returns an `int`
(let [fn- (com.sun.jna.Function/getFunction "some-lib" "quux")]
  (fn [& args]
    (.invoke fn- Integer (to-array args))))

The goal is to get a type to substitute for Integer above that will wrap MyStruct as a value.

The only resource that I've found covering this subject is this article, but it only discusses how to pass structs by reference.

Given that, here are the different approaches I've tried to take to solve this problem:

Create a class that inherits from Structure, which is JNA's built-in mechanism for creating and using structs. Given the information on that page, I tried to create the following class using only Clojure:
```
class MyStruct extends Structure implements Structure.ByValue {
    int foo;
    int bar;
    double baz;
}
```
deftype doesn't work for this scenario, since the class needs to inherit from the abstract class Structure, and gen-class doesn't work because the class needs to have the public non-static fields foo, bar and baz.

From what I can tell, none of the standard Clojure Java interop facilities can create the above class.
Create a class that inherits from Structure and override the struct field getter/setter methods. Since gen-class is (I believe) the only Clojure construct that allows for direct inheritance, and it doesn't support multiple public non-static fields, the next alternative is to simply not use fields at all. Looking at the Structure abstract class documentation, it seems like there's a concoction of overrides possible to use 'virtual' struct fields, such that it really gets and sets data from a different source (such as the state field from gen-class). Looking through the documentation, it seems like overriding readField, writeField, and some other methods may have the intended effect, but the I was unclear how to do this from reading the documentation, and I couldn't find any similar examples online.
Use a different storage class. JNA has a myriad of classes for wrapping native types. I'm wondering if, rather than defining and using a Structure class, I could use another generic class that can take an arbitrary number of bits (like how Integer can hold anything that's 4 bits wide, regardless of what type the source 'actually' is). Is it possible to, for example, say that a function returns an array of bytes of length 16 (since sizeof(MyStruct) is 16)? What about wrapping a fixed-size array in a container class that implements NativeMapped? I couldn't find examples of how to do either.

JNA provides block memory allocation in the form of com.sun.jna.Memory. From there, you can extract arbitrary types from arbitrary offsets in memory using Pointer.getXXX() methods. — technomage
@technomage Using Memory as a return type for createStruct returns a Pointer, whose address is 0x0000000400000003. In other words, it treats the raw memory value of the struct as a pointer. When dereferencing (through any of the read or getXXX methods), it crashes the VM. — Kyle Lacy
The problem is that struct parameter and return value semantics vary by compiler and the Structure layout is actually used by the native code to determine how to set up the stack properly. I was suggesting you might be able to use Memory to assign an arbitrary block of memory to an existing Structure.ByValue instance. — technomage
You might be better off asking the clojure folks. JNA requires the Structure.ByValue tag on both the actual argument and the library function mapping signature in order to pass a structure by value on to the native code. — technomage
If you do somehow get around defining public fields, you'd need to override the (package-protected) getTypeInfo() for your structure, which returns FFI type information for your structure (which is itself a structure). — technomage

Kyle Lacy Kyle Lacy · Accepted Answer · 2014-07-29T05:01:13

Clojure actually embeds a forked version of ASM, which is used for generating and loading JVM bytecode.

On top of this library, I built small DSL (that's incomplete and probably broken) that allows for creating arbitrary Java classes, with a Java-like syntax. Currently, it's just a GitHub gist, and only supports extending classes, implementing interfaces, adding constructors that call superclass constructors, and fields.

Here's the solution to the above question, using this DSL:

(def-class MyStruct :extends    com.sun.jna.Structure
                    :implements [com.sun.jna.Structure$ByValue]
  ; Declare all the constructors, which just forward their
  ; arguments to the constructors of com.sun.jna.Structure
  (com.sun.jna.Structure [])
  (com.sun.jna.Structure [com.sun.jna.TypeMapper])
  (com.sun.jna.Structure [Integer])
  (com.sun.jna.Structure [Integer com.sun.jna.TypeMapper])
  (com.sun.jna.Structure [com.sun.jna.Pointer])
  (com.sun.jna.Structure [com.sun.jna.Pointer Integer])
  (com.sun.jna.Structure [com.sun.jna.Pointer Integer com.sun.jna.TypeMapper])

  ; Declare the fields of the struct
  ^Integer foo
  ^Integer bar
  ^Double  baz)

Now, I can do the following:

(defn createStruct [& args]
  (let [fn- (com.sun.jna.Function/getFunction "some-lib" "createStruct")]
    (.invoke fn- MyStruct (to-array args))))

(defn addStruct [& args]
  (let [fn- (com.sun.jna.Function/getFunction "some-lib" "addStruct")]
    (.invoke fn- Double (to-array args))))

(addStruct (createStruct))
; => 10.14

Getting and passing structs by value in Clojure with JNA

1 Answers