react-native start gives me "Watchman error: too many pending cache jobs"

2
votes

Problem

When I create a bundle - the watchman crawl initially fails with "too many pending cache jobs", which seems to slow down the bundling process:

$ react-native start --reset-cache

Loading dependency graph...jest-haste-map: Watchman crawl failed. Retrying once with node crawler.

Usually this happens when watchman isn't running. Create an empty .watchmanconfig file in your project's root folder or initialize a git or hg repository in your project.

Error: Watchman error: too many pending cache jobs. Make sure watchman is running for this project. See https://facebook.github.io/watchman/docs/troubleshooting.html

Also, these may be related:

  • I can get rid of the error by excluding node_modules from the watch config
  • Without the error, and the bundler starts in 20 seconds
  • With the error, the craw takes 4 seconds, but the bundler starts in 4 minutes

Background

Pre-bundle scripts

I run these before the bundler, to attempt to get more stability:

watchman watch-del-all
watchman shutdown-server
sudo sysctl -w kern.maxfiles=5242880
sudo sysctl -w kern.maxfilesperproc=524288

Watchman config file

I set my .watchconfig file to be extra permissable:

{
  "ignore_dirs": [],
  "fsevents_latency": 0.5,
  "fsevents_try_resync": true
}

Watchman log

The crawl does complete, within a few seconds according to the full log

$ tail -f /usr/local/var/run/watchman/coolman-state/log

22:08:24,193: [client=0x7ff4a9686d98:stm=0x7ff4a9b2de00:pid=0] send_error_response: too many pending cache jobs
22:08:24,340: [listener] Watchman 4.9.0 <no build info set> starting up on coolman
22:08:24,341: [listener] path /Users/coolman/project/react-native-app is on filesystem type apfs
22:08:24,342: [listener] root /Users/coolman/project/react-native-app using watcher mechanism fsevents (auto was requested)
22:08:24,344: [listener] file limit is 2560 kern.maxfilesperproc=524288
22:08:24,344: [listener] raised file limit to 524288
22:08:24,345: [listener] launchd: "sock" wasn't present in Sockets
22:08:28,805: [io 0x7f87ba000818 /Users/coolman/project/react-native-app] PERF: {"ru_nvcsw": 1460, "ru_nsignals": 0, "ru_msgrcv": 0, "ru_msgsnd": 0, "ru_inblock": 0, "ru_majflt": 0, "ru_nswap": 0, "ru_minflt": 28103, "ru_ixrss": 0, "ru_maxrss": 115109888, "system_time": 2.9349970000000001, "user_time": 1.327971, "elapsed_time": 4.4614079999999996, "pid": 29414, "ru_idrss": 0, "meta": {"root": {"watcher": "fsevents", "ticks": 2, "number": 1, "case_sensitive": false, "recrawl_count": 0, "path": "/Users/coolman/project/react-native-app"}}, "version": "4.9.0", "ru_oublock": 0, "ru_nivcsw": 2171, "start_time": 1564952904.3444469, "description": "full-crawl"}
22:08:28,805: [io 0x7f87ba000818 /Users/coolman/project/react-native-app] crawl complete
1
react-nativewatchman

1 Answers

1
votes

Solution

Adding "content_hash_max_items": 500000 to my .watchmanconfig solved it for me.

Explanation:

Sounds like you need to tune the metadata cache size(s) for your workload; that error occurs when watchman is attempting to insert more cache entries than are configured.

You can specify larger values in your .watchmanconfig file:

  • content_hash_max_items is related to the content.sha1hex metadata and defaults to 131072. A related option is content_hash_max_warm_per_settle which defaults to 1024 and which causes up to this many items to be assessed on each settle.
  • symlink_target_max_items defaults to 32768. This cache is used for readlink in the case that queries ask for the symlink target. This is less likely to be your issue.

I would suggest running find /Users/coolman/project/react-native-app | wc -l to get an upper bound on the number of files currently in your project and use that to size these caches; if you have more files than those defaults then you should raise those cache settings accordingly.

I do not recommend changing the fsevents_latency parameter from the default; can you share more context on what led you to change it?

Github