Can nodejs worker threads be used for executing long running file I/O based javascript code?

Question

I can see that NodeJS is bringing in multi-threading support via its worker threads module. My current assumption (I have not yet explored personally) is that I can offload a long running /cpu intensive operation to these worker threads.

I want to understand the behaviour if this long running piece of code has some intermittent event callbacks or chain of promises. Do these callbacks still execute on the worker threads, or do they get passed on back to the main thread?

If these promises come back to main thread, the advantage of executing the worker thread may be lost.

Can someone clarify?

Update => Some context of the question

I have a http req that initiates some background processing and returns a 202 status. After receiving such request, I am starting a background processing via

setTimeout (function() { // performs long running file read operations.. })

and immediately return a 202 to the caller.

However, I have observed that, during this time while this background operation is going, other http requests are either not being processed, or very very sluggish at the best.

My hypothesis is that this continuous I/O processing of a million+ lines is filling up the event loop with callbacks / promises that the main thread is unable to process other pending I/O tasks such as accepting new requests.

I have explored the nodejs cluster option and this works well, as the long task is delegated to one of the child processes, and other instances of cluster are available to take up additional requests.

But I was thinking that worker threads might solve the same problem, without the overhead of cloning the process.

Answer 1

I assume each worker thread would have its own event loop.

So if you emit an event in a worker thread, only that thread would receive it and trigger the callback. The same for promises, if you create a promise within a worker, it will only be resolved by that worker.

This is supported by their statement in the documentation regarding Class: Worker: Most Node.js APIs are available inside of it (with some exceptions that are not related to event processing).

However they mention this earlier in the docs:

Workers are useful for performing CPU-intensive JavaScript operations; do not use them for I/O, since Node.js’s built-in mechanisms for performing operations asynchronously already treat it more efficiently than Worker threads can.

I think some small scale async code in worker threads would be fine, but having more callbacks/promises would hurt performance. Some benchmarks could shed some light on this.