The required JS dependency "readable-stream/writable.js" is not available, it was required by "node_modules/stream-browserify/index.js"

Question

I am getting this error when trying to build a Clojurescript project with shadow-cljs. I've tried looking for syntax errors as described here but I can get the same the error with a single line and a single import although not all imports cause the same error.

This compiles:

(ns campfire.core)

(defn init [] (println "ok"))

This doesn't:

(ns campfire.core
  (:require ["bugout" :as b]))

(defn init [] (println "ok"))

The output from the above example is:

shadow-cljs - config: /home/ru/Projects/campfire/shadow-cljs.edn
shadow-cljs - HTTP server available at http://localhost:3000
shadow-cljs - server version: 2.11.18 running at http://localhost:9630
shadow-cljs - nREPL server started on port 8777
shadow-cljs - watching build :frontend
[:frontend] Configuring build.
[:frontend] Compiling ...
[:frontend] Build failure:
The required JS dependency "readable-stream/writable.js" is not available, it was required by "node_modules/stream-browserify/index.js".

Dependency Trace:
        campfire/core.cljs
        node_modules/bugout/index.js
        node_modules/bs58check/index.js
        node_modules/create-hash/browser.js
        node_modules/cipher-base/index.js
        node_modules/stream-browserify/index.js

Searched for npm packages in:
        /home/ru/Projects/campfire/node_modules

See: https://shadow-cljs.github.io/docs/UsersGuide.html#npm-install

package.json

{
  "name": "campfire",
  "version": "0.0.1",
  "private": true,
  "scripts": {
    "build": "shadow-cljs release frontend"
  },
  "devDependencies": {
    "shadow-cljs": "2.11.18"
  },
  "dependencies": {
    "bugout": "^0.0.10",
    "webtorrent": "^0.114.1"
  }
}

shadow-cljs.edn

{:source-paths
 ["src/dev"
  "src/main"
  "src/test"]

 :dependencies
 []

 :dev-http {3000 "public"}
 :nrepl {:port 8777}
 :builds
 {:frontend
  {:target  :browser
   :modules {:main {:init-fn campfire.core/init}}}}}

I've seen similar build errors that were fixed by clearing .shadow-cljs etc and rebuilding but nothing like that seems to be helping. I'm new to shadow so apologies if this is something obvious. Does anyone have any idea what's going on here?

Update

So it looks like what's happening is that stream-browserify 2.0.2 requires readable stream ^2.0.2 which npm installs in the nested node_modules folder. Elsewhere readable-stream 3.6.0 is being installed in top level node_modules. Shadow is trying to resolve writer.js against the 3.6.0 version of readable stream instead of the 2.0.2 version.

Confusingly though, stream-browserify isn't a dependency of cipher-base as given in the dependency trace but of node-libs-browser which is itself a dependency of shadow-cljs.

Is it possible that this is a bug in shadow or is it expected behaviour?

Update 2

I've created an example repo that replicates what I'm seeing as simply as I can here.

Thomas Heller Thomas Heller · Accepted Answer · 2021-02-22T08:51:00

Did you actually install the shadow-cljs dependency in the project? Does the directory node_modules/shadow-cljs exist?

I see it listed in devDependencies so it should be installed but it might not be if you never actually called npm install in the project or npm is set to production mode which won't install devDependencies. All of this is part of the node-libs-browser package which seems to be missing as well and should have been installed due to being a dependency of shadow-cljs.

The required JS dependency "readable-stream/writable.js" is not available, it was required by "node_modules/stream-browserify/index.js"

Update

Update 2

2 Answers