metro-file-map: Do not miss files changed while the file map builds

[cpruijsen]

Summary

Fixes #1954: files created or modified while the file map builds are no longer missed until the next restart.

FileMap.build() crawls, applies the delta, persists the cache and only then starts the watcher. A change made after the crawl reads a file, but before the watcher listens, is lost. This happens with Watchman and with the native watcher. The fix follows the approach suggested in the issue:

• Watchman: #watch() now passes the crawl's clocks to Watcher.watch(), and WatchmanWatcher subscribes since the crawl clock for its watch root instead of requesting a fresh clock. Changes in the gap arrive in the initial subscription result. Crawl clocks are keyed relative to rootDir, but a backend learns its watch root only from watch-project, so Watcher re-keys them by absolute (posix-separated) watch root, the inverse of what the Watchman crawler does. With no matching clock, or on a reconnect, the old fresh-clock path is unchanged.
• Native and fallback: these have no clock, so in watch mode Watcher starts them before the crawl. Their events are buffered until watch() attaches, then replayed in order through the normal change handler, after the health-check and recrawl filters. A replayed event for a file the crawl already saw becomes an addOrModify with fresher metadata. If end() aborts the build while these backends start, they are closed.

Judgement call: Watchman keeps its lazy start, because subscribing since the crawl clock already covers the window. The alternatives are to start every backend before the crawl, or to keep the lazy start and recrawl after subscribing. Both are more code, and the recrawl leaves a narrower window instead of none. I can switch if you prefer one of them.

Not covered: a Watchman reconnect still re-subscribes from a fresh clock, and a Watchman crawl that falls back to the node crawler has no clock to pass. @expo/metro-file-map needs the same change separately.

Changelog: [Fix] Fix files created or modified while Metro starts up being missed until restart

Test plan

New tests in WatchmanWatcher-test.js (subscription is since a supplied clock, with no clock command) and index-test.js (clocks reach the watcher keyed by absolute watch root, and an event emitted during the node crawl reaches the file system and the change event) fail without this change and pass with it. A third test, that a clock for a different root keeps the fresh-clock path, passes before and after, as intended.

[vzaidman]

c.c @robhogan

[robhogan]

Thanks! Approved with a small refactor I've added around useWatchman.

useWatchman for the watcher really means "Watchman is running and has the required capabilities, and the user hasn't explicitly disabled it, so try to use it"

But there are still common cases where it can't be used - e.g. a watch root is a network share, or watchman can't find a watchman root.

In those cases we fall back to Node crawler, and originally in this PR that was a gap, because we'd perform a node crawl before starting to watch.

The change I've pushed keeps most paths identical to yours, but reshuffles internals so that we can start buffered-watch node crawl in the fallback case, as well as the !useWatchman case.

[robhogan]

Bit more testing after merging in main (mainly addRoot and related).

Testing (AI-driven): an end-to-end test against a real FileMap, real watcher backends and real Watchman. It adds, edits and deletes files after the crawl has read them but before the watcher attaches, then checks the file map matches the disk and every change was emitted. Run on macOS, Linux (also on Node 22.13.0) and Windows 11, each with Watchman, useWatchman: false, Watchman not installed and a failing Watchman crawl, plus addRoot in each mode. Two layouts: the project root as the Watchman watch root, and a parent watch root with a sibling watch folder.

• Everything passes on this PR. On main the same test fails 8 of 11 cases, on every OS.
• A Watchman recrawl in the gap is handled precisely.
• metro-file-map and node-haste suites pass on all three. The one Windows failure (Watchman reporting a symlink's size as 0) fails identically on main.

To follow: if Watchman loses the watch or restarts between the crawl and the subscription, the first result is a fresh instance, so every file is replayed as a change and deletes in the gap are still missed. That's still strictly better than main, which misses everything in the gap, so it isn't blocking. We can follow up with empty_on_fresh_instance and a recrawl.