How to use worker_threads on Sapper?

Question

I'm trying to start a thread on Sapper using the nodejs worker_threads, but it looks like it's trying to start also another instance of Polka.

You can find my code on this repo, but it basically has a route to http://localhost:3000/worker that fetches the server-side worker\run.js. It calls the worker.js that has the worker_threads logic.

The run.js calls the main() function on worker.js and creates a new thread that re-loads itself inside a worker thread instance, this time matching the if (!isMainThread) {...} condition, executing the console.logs (I tried other uses and the same problem appeared).

/* worker.js */

import { Worker, isMainThread } from 'worker_threads';

export const main = () => new Promise(function (resolve, reject) {
  if (!isMainThread) return;

  // This re-loads the current file inside a Worker instance.
  const wk = new Worker(__filename);
  wk.on("online", () => console.log("Worker UP"));
  wk.on("message", (msg) => {
    console.log("message ~>", msg);
    resolve(msg);
  });
  wk.on("exit", (code) => console.warn("exit ~>", code));
  wk.on("error", (err) => {
    console.error("error ~>", err);
    reject(err);
  });
})

if (!isMainThread) {
  console.log('Inside Worker!');
  console.log(isMainThread);  
}

But it returns the following error (note that the console.log(isMainThread) is not executed):

> sapper dev

✔ client (3.3s)
✔ server (3.3s)
> Listening on http://localhost:3000
✔ service worker (84ms)
Worker UP
Inside Worker!
error ~> Error: listen EADDRINUSE: address already in use :::3000
    at Server.setupListenHandle [as _listen2] (net.js:1313:16)
    at listenInCluster (net.js:1361:12)
    at Server.listen (net.js:1449:7)
    at Polka.listen (/mnt/disk2/adr/lab2/pollers/worker/node_modules/.pnpm/registry.npmjs.org/polka/1.0.0-next.11/node_modules/polka/build.js:59:22)
    at Object.<anonymous> (/mnt/disk2/adr/lab2/pollers/worker/__sapper__/dev/server/server.js:3119:3)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1153:10)
    at Module.load (internal/modules/cjs/loader.js:977:32)
    at Function.Module._load (internal/modules/cjs/loader.js:877:14)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:74:12) {
  code: 'EADDRINUSE',
  errno: 'EADDRINUSE',
  syscall: 'listen',
  address: '::',
  port: 3000
}
exit ~> 1

edit:

After analyzing the output from rollup:

/* __sapper__/dev/server/server.js */

'use strict';

function _interopDefault (ex) { return (ex && (typeof ex === 'object') && 'default' in ex) ? ex['default'] : ex; }

var sirv = _interopDefault(require('sirv'));
var polka = _interopDefault(require('polka'));
var compression = _interopDefault(require('compression'));
var fs = _interopDefault(require('fs'));
var path = _interopDefault(require('path'));
var worker_threads = require('worker_threads');
var Stream = _interopDefault(require('stream'));
var http = _interopDefault(require('http'));
var Url = _interopDefault(require('url'));
var https = _interopDefault(require('https'));
var zlib = _interopDefault(require('zlib'));

if (!worker_threads.isMainThread) {
    console.log('Inside Worker!');
    console.log(worker_threads.isMainThread);  
}

var route_0 = /*#__PURE__*/Object.freeze({
    __proto__: null
});

const main = () => new Promise(function (resolve, reject) {
  if (!worker_threads.isMainThread) return;

  // This re-loads the current file inside a Worker instance.
  const wk = new worker_threads.Worker(path.join(__dirname, "threads.js"));
  wk.on("online", () => console.log("Worker UP"));
  wk.on("message", (msg) => {
    console.log("message ~>", msg);
    resolve(msg);
  });
  wk.on("exit", (code) => console.warn("exit ~>", code));
  wk.on("error", (err) => {
    console.error("error ~>", err);
    reject(err);
  });
});
...

Rollup bundles all files into a single one, hence the Worker(__filename) loads it again in a thread.

So I put the file to be loaded in a thread into a separated file:

/* thread.js */

import { isMainThread } from 'worker_threads';

if (!isMainThread) {
    console.log('Inside Worker!');
    console.log(isMainThread);
}

And changed the construction of the Worker instance to:

const wk = new Worker(path.join(__dirname, "thread.js"));

Now the error has changed to:

"Cannot find module '/mnt/disk2/adr/lab2/pollers/worker/__sapper__/dev/server/thread.js'"

The problem now boils down to a rollup config to keep the thread.js separated from the server.js.

Sapper handles the rollup config in its own way, otherwise I could set the rollup.config.js like this:

export default [
  {
    input: "server.js",
    output: {
      file: "dist/server.js",
      format: "cjs",
    },
  },
  {
    input: "thread.js",
    output: {
      file: "dist/thread.js",
      format: "cjs",
    },
  },
];

Now the question is: is there a way to do that without modifying Sapper code?

Adriano P Adriano P · Accepted Answer · 2020-06-06T02:19:35

To solve this, at least on sapper v0.27.0, I added the file I wanted to be executed as a thread in the server.input on rollup.config.js:

export default {

  server: {
    input: { ...config.server.input(), threads: "src/routes/worker/threads.js" },
    output: config.server.output(),
  }
}

config.server.intput() returns { server: `${__chunk_3.src}/server.js` }, then adding threads: "path_to_thread_file" did the trick.

How to use worker_threads on Sapper?

edit:

1 Answers