How to statically link node.js in rust?

4
votes

I would like to embed node.js in Rust. I'm not interested in writing a node.js addon with NAPI or controlling node.js extensively from within rust. All I need is the node.js main() start method - the equivalent of doing node myscript.js.

Why? I am building a self-contained single-file binary desktop application in Rust and would like to run node.js scripts with an embeded node.js runtime. Node.js is not guaranteed to be on the end user's computer, and startup time is sensitive so extracting a self-contained zip of node.js from within the binary to the filesystem is undesirable.

I believe I am having problems with statically linking node.js in my rust (binary) project.

I pull down the nodejs source code

git clone https://github.com/nodejs/node

And rename the main method in node_main.cc

sed -i .bak "s/int main(/int node_main(/g" ./src/node_main.cc

Then I build node.js as static libraries

./configure --enable-static
make -j4

I have a c++ wrapper file wrapper.cpp to expose the node_main() method via extern c

wrapper.cpp:

#include <string>
#include <iostream>

using namespace std;

int node_main(int argc, char **argv);

extern "C" {
  void run_node() {
    cout << "hello there! general kenobi..." << endl;
    char *args[] = { (char*)"tester.js", NULL };
    node_main(1, args);
  }
}

At this point I was able to successfully build the c++ wrapper as a binary while statically linking node.js libs and run node.js from c++ successfully. However, from rust...

main.rs:

extern {
  fn run_node();
}

fn main() {
  println!("hey there this is RUST");
  unsafe { run_node(); }
}

build.rs:

extern crate cc;

fn main() {
  println!("cargo:rustc-link-search=native=../node/out/Release");

  println!("cargo:rustc-link-lib=static=node");
  println!("cargo:rustc-link-lib=static=uv");

  println!("cargo:rustc-link-lib=static=v8_base");
  println!("cargo:rustc-link-lib=static=v8_libbase");
  println!("cargo:rustc-link-lib=static=v8_snapshot");
  println!("cargo:rustc-link-lib=static=v8_libplatform");

  println!("cargo:rustc-link-lib=static=icuucx");
  println!("cargo:rustc-link-lib=static=icui18n");
  println!("cargo:rustc-link-lib=static=icudata");
  println!("cargo:rustc-link-lib=static=icustubdata");

  println!("cargo:rustc-link-lib=static=brotli");
  println!("cargo:rustc-link-lib=static=nghttp2");

  cc::Build::new()
    .cpp(true)
    .file("wrapper.cpp")
    .compile("libwrapper.a");
}

Note the rustc-link-search path is relative above.

When I run cargo build I get the following error:

= note: Undefined symbols for architecture x86_64:
          "node_main(int, char**)", referenced from:
              _run_node in libwrapper.a(wrapper.o)
        ld: symbol(s) not found for architecture x86_64
        clang: error: linker command failed with exit code 1 (use -v to see invocation)

I'm not sure if linker order is important, but I tried some different ordering combinations with no luck. I also tried linking all the lib .a files from node/out/Release but no difference. I also tried combining all the .a files from node/out/Release into one lib .a file but received duplicate symbol errors. I tried different tags of node.js (like v11.15.0) with no difference.

I am doing this on MacOS 10.14.5

$ rustc --version
rustc 1.36.0 (a53f9df32 2019-07-03)
$ cargo --version
cargo 1.36.0 (c4fcfb725 2019-05-15)
$ g++ --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Devel
oper/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.1 (clang-1001.0.46.4)
Target: x86_64-apple-darwin18.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

Cargo.toml:

[build-dependencies]
cc = "1.0"

I'm open to better ways of embedding node.js in rust if there are any good ideas.

1
c++node.jsruststatic-linking

1 Answers

3
votes

I found a number of issues that once resolved allows successful static linking of node.js in Rust.

node_main.cc is not included in the libnode.a output

I found this by using nm to lookup symbols. I was not able to find node_main() and neither node_main.o in the symbols list. Thus I realized that nothing in node_main.cc would ever be exported.

Fix: Expose a library entry point C function in another file like node.cc. Note that here we are adding a completely new function that calls node::Start()

node.cc

extern "C" int node_main(int argc, char** argv) {
    return node::Start(argc, argv);
}

need extern "C" because of c++ name symbol mangling

Again, using the nm tool to search through all symbols in the libnode.a file I was able to discover that node_main() function symbol was mangled. In order to find this symbol from Rust it must not be mangled, and this is accomplished with extern "C"

Fix: make sure to prepend functions intended to be exposed to Rust with extern "C"

node.cc

extern "C" int node_main()

depending on node version, some additional symbols are missing

According to node.js Github issue #27431 some symbol stubs do not get output in the static library. I was working with node.js v13.x.x tags where this is an issue, so I had to create libraries for these extra stubs and link them in the rust build configuration.

Fix: Create static libraries from the stubs and link them in build.rs

ar rcs obj/Release/lib_stub_code_cache.a obj/Release/obj.target/cctest/src/node_code_cache_stub.o
ar rcs obj/Release/lib_stub_snapshot.a obj/Release/obj.target/cctest/src/node_snapshot_stub.o

Final result

building node.js

git clone https://github.com/nodejs/node
cd node
printf 'extern "C" int node_main(int argc, char** argv) { return node::Start(argc, argv); }' >> src/node.cc
./configure --enable-static
make -j4

# temporary fix: https://github.com/nodejs/node/issues/27431#issuecomment-487288275
REL=obj/Release
STUBS=$REL/obj.target/cctest/src
ar rcs "$REL/lib_stub_code_cache.a $STUBS/node_code_cache_stub.o"
ar rcs "$REL/lib_stub_snapshot.a $STUBS/node_snapshot_stub.o"

wrapper.cpp

#include <string>
#include <iostream>
using namespace std;

extern "C" {
  int node_main(int argc, char** argv);

  void run_node() {
    cout << "hello there! general kenobi...\n";
    char *args[] = { (char*)"tester.js", NULL };
    node_main(1, args);
  }
}

build.rs

extern crate cc;

fn main() {
  cc::Build::new()
    .cpp(true)
    .file("wrapper.cpp")
    .compile("libwrapper.a");

  println!("cargo:rustc-link-search=native=../node/out/Release");

  println!("cargo:rustc-link-lib=static=node");
  println!("cargo:rustc-link-lib=static=uv");

  // temporary fix - https://github.com/nodejs/node/issues/27431#issuecomment-487288275
  println!("cargo:rustc-link-lib=static=_stub_code_cache");
  println!("cargo:rustc-link-lib=static=_stub_snapshot");
  // end temporary fix

  println!("cargo:rustc-link-lib=static=v8_base_without_compiler");
  println!("cargo:rustc-link-lib=static=v8_compiler");
  println!("cargo:rustc-link-lib=static=v8_initializers");
  println!("cargo:rustc-link-lib=static=v8_libbase");
  println!("cargo:rustc-link-lib=static=v8_libplatform");
  println!("cargo:rustc-link-lib=static=v8_libsampler");
  println!("cargo:rustc-link-lib=static=v8_snapshot");

  println!("cargo:rustc-link-lib=static=icuucx");
  println!("cargo:rustc-link-lib=static=icui18n");
  println!("cargo:rustc-link-lib=static=icudata");
  println!("cargo:rustc-link-lib=static=icustubdata");

  println!("cargo:rustc-link-lib=static=zlib");
  println!("cargo:rustc-link-lib=static=brotli");
  println!("cargo:rustc-link-lib=static=cares");
  println!("cargo:rustc-link-lib=static=histogram");
  println!("cargo:rustc-link-lib=static=http_parser");
  println!("cargo:rustc-link-lib=static=llhttp");
  println!("cargo:rustc-link-lib=static=nghttp2");
  println!("cargo:rustc-link-lib=static=openssl");
}

main.rs

extern {
  fn run_node();
}

fn main() {
  println!("a surprise to be sure, but a welcome one");
  unsafe { run_node(); }
}