Skip to content

fix(runner): replace chokidar with native fs.watch in the test watcher - #43716

Merged
Dmitry Gozman (dgozman) merged 2 commits into
microsoft:mainfrom
dgozman:fix-43653
Oct 29, 2026
Merged

Dmitry Gozman (dgozman) merged 2 commits into
microsoft:mainfrom
dgozman:fix-43653

Conversation

@dgozman

@dgozman Dmitry Gozman (dgozman) commented Oct 28, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • FSWatcher in packages/playwright/src/runner/fsWatcher.ts now uses fs.watch instead of chokidar.
  • On macOS and Windows, each testDir is watched with one recursive fs.watch handle.
  • On Linux and other platforms, each directory under testDir is watched with one non-recursive fs.watch handle. node_modules, project output folders and symlinked directories are not entered.
  • Dependencies of watched test files that live outside testDir are watched through their parent directory, so atomic saves (write to a temp file, then rename) keep working.
  • Directories passed to the watch request are watched recursively. The VS Code extension passes the testDir when watching all tests, and a folder when watching a folder.
  • The playwright package no longer imports chokidar. chokidar stays in playwright-core for the server registry and in the build scripts.

Fixes #43653

Why

UI mode calls initialize({ watchTestDirs: true }) so the test tree refreshes when test files are added, changed or removed. The runner watches every project testDir from the start, before the user enables the watch toggle. Since #42238 (1.63) we ship chokidar 4, which dropped FSEvents support. Chokidar 4 calls fs.watch once per file and once per directory. On macOS a non-recursive fs.watch is a kqueue watch and needs one open descriptor per watched path. A testDir with more than about 10k files pushes the process past OPEN_MAX (10240), and fork() for the next worker fails with spawn EBADF.

Chokidar's ignored option does not help here: it only prunes the tree, and every file that is not ignored still gets its own watch.

chokidar vs fs.watch

What the chokidar README lists as its advantages over raw fs.watch, and whether we use them:

chokidar advantage Do we rely on it?
Events are reported as add / change / unlink instead of rename No. The runner only reads the path and ignores the event kind.
macOS events report filenames, events are not reported twice Recursive fs.watch on macOS uses FSEvents and reports filenames. Duplicates are already handled by our 250 ms batching and the consumer re-lists tests anyway.
atomic option for editors that write a temp file and rename We never set it. Watching directories instead of files handles renames naturally.
awaitWriteFinish for chunked writes We never set it in the runner. Only serverRegistry.ts uses it, and it stays on chokidar.
File / dir filtering with ignored We filter by path in our code.
Symbolic links are followed The test loader never follows symlinks (see below), so a watcher that follows them has nothing to report.
Recursive watching is always supported Written against Node 14. We require Node 20+, where fs.watch({ recursive: true }) is supported on Linux, macOS and Windows.

Pros and cons of fs.watch

Pros:

  • macOS: one FSEvents handle per testDir instead of one descriptor per file. Fixes the issue.
  • Windows: one ReadDirectoryChangesW handle per testDir instead of one handle per file and directory.
  • Linux: one inotify watch per directory instead of one per file and directory. Uses far fewer of fs.inotify.max_user_watches.
  • Directory watches report renames into the directory, so atomic saves from editors are caught without per-file re-watching.
  • No chokidar import in the playwright package. The watcher is about the same size as before.

Cons:

  • Linux needs our own directory walker instead of fs.watch({ recursive: true }). See below.
  • Symlinked directories inside testDir are not entered. This matches the loader.
  • Events from .git and other non-test folders inside testDir arrive and are dropped by path in our code instead of being pruned up front. This was already the case with chokidar's per-file watches.
  • filename can be null in rare cases. We then report the watched root, which makes the UI re-list tests.

Measurements on macOS

Apple M5 Pro, macOS 26.7.1, Node.js 26.7.0. One testDir with a.test.ts and N files in src/dir-K/ folders of 100 files each. The old FSWatcher (chokidar 4, as on main) and the new one (fs.watch) run in a fresh process each, watching the testDir the way the runner does. Latency is the time from writeFileSync to the runner callback and includes the 250 ms batching that both have, so raw event delivery is about 2 ms for chokidar and about 13 ms for FSEvents. Descriptors are counted in /dev/fd. Each cell is the median of 5 changes to a top-level file; changes to a file deep in the tree and new files were within 15 ms of these.

Node raises the soft descriptor limit to the hard limit at startup, so what matters is the hard limit. The macOS default is OPEN_MAX = 10240.

Files in testDir Watcher Setup Open descriptors while watching RSS growth Latency Close
1,000 chokidar 160 ms 1,013 24 MB 252 ms 15 ms
1,000 fs.watch 1 ms 12 1 MB 263 ms 0 ms
5,000 chokidar 1.9 s 5,013 85 MB 252 ms 41 ms
5,000 fs.watch 1 ms 12 1 MB 263 ms 0 ms
10,000 chokidar 4.3 s 10,013 112 MB 253 ms 108 ms
10,000 fs.watch 1 ms 12 1 MB 263 ms 1 ms
20,000 chokidar 9.6 s 20,013 154 MB 253 ms 225 ms
20,000 fs.watch 1 ms 12 1 MB 263 ms 0 ms
50,000 chokidar 32.9 s 50,013 262 MB 252 ms 897 ms
50,000 fs.watch 1 ms 12 1 MB 263 ms 1 ms

The chokidar rows above 10,000 files were measured with the hard limit raised to 1,048,576. With the default hard limit of 10240, chokidar on 20,000 files prints (libuv) kqueue(): Too many open files and the process can no longer open stdout (Error: open EMFILE). This is the failure from #43653, where it surfaced as spawn EBADF for the next worker. The new watcher stays at 12 descriptors at every size, because FSEvents does not use a descriptor per path.

A change made right after update() returned, before the watcher had time to settle, was reported by both watchers in every run, so the asynchronous start of the FSEvents stream did not drop events here.

Linux is not covered by these numbers. There the new watcher uses one inotify watch per directory, see below.

Why Linux gets its own walker

Linux inotify has no "watch the subtree" flag. A watch covers one directory and reports events for its direct entries only. libuv only wraps what the kernel offers: uv_fs_event_start in src/unix/linux.c ignores the flags argument, and the libuv docs state that UV_FS_EVENT_RECURSIVE is supported only on macOS and Windows. Up to Node 18, fs.watch with recursive: true threw ERR_FEATURE_UNAVAILABLE_ON_PLATFORM on Linux.

Node 19.1 made the call succeed by adding a JavaScript fallback, lib/internal/fs/recursive_watch.js, which is unchanged from Node 20 through 26. It reads the tree with readdirSync and, for every entry, calls statSync and a non-recursive fs.watch. That is one inotify watch per file and one per directory, and nothing is skipped. The ignore option that prunes subtrees arrived in Node 26, and we support Node 20.

So on Linux the Node fallback has the same cost as chokidar 4, and is worse when testDir contains node_modules, which chokidar pruned. For this repository that would mean about 69,000 watches and stat calls at startup, against a fs.inotify.max_user_watches budget that is 8,192 on older kernels and 65,536 or 524,288 on recent distributions. Running out raises ENOSPC and the watch for that root is lost.

An inotify watch on a directory already reports creation, modification, deletion and rename of its direct children with the child's name, and the runner only needs the path. So on Linux we read the tree once, watch each directory, skip node_modules, output folders and symlinked directories, add a watch when an event names a new directory, and close the watches under a directory that disappeared. For this repository that is about 6,700 watches. Once Node 26 is the minimum, the walker can be replaced by fs.watch({ recursive: true, ignore }).

Symlink support history

  • The test loader (collectFiles in projectUtils.ts) reads directories with withFileTypes and keeps an entry only when isFile() or isDirectory() is true. A symlink answers false to both, so symlinked test files and symlinked directories are skipped.
  • [Bug]: Symlinked directories inside testDir are ignored in MacOS #32140 "Symlinked directories inside testDir are ignored in MacOS" (Aug 2024). A fix in fix(runner): enhance file collection to handle symbolic links #38073 was declined because of the risk of endless loops through directory links, and the issue was closed in Nov 2025 with the suggestion to use tags or separate projects. A later comment reported that Bazel runfiles trees consist of file symlinks, so sharded runs collected zero tests and stayed green; following file symlinks only was proposed.
  • [Bug]: VSCode extension does not recognize Symlink directories #33317 "VSCode extension does not recognize Symlink directories" (Oct 2024) was about a project root reached through a symlinked path under WSL, not about test discovery. Closed for low activity.

Since the loader does not follow symlinks, the watcher not following them is consistent, not a regression. If file symlinks are ever supported by the loader, the dependency tracking already gives the watcher the real target paths of imported files.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

🔴 CI failures are caused by this PR

Hi, I'm the Playwright bot. I checked the latest CI report and compared the old and new watchers.

Details

At 5e3cb472a6c1, the 20 reported failures fall into two PR-caused groups.

Caused by this PR — new test setup (2 failures)

[playwright-test] › ui-mode-test-watch.spec.ts:61 › should watch with many files in testDir fails on Windows and macOS, on the initial attempt and all three retries. Each raises EMFILE: too many open files at playwright-test-fixtures.ts:87, while creating the files, before the UI runner or watcher starts. The new test supplies 11,000 files to a fixture that writes them through an unbounded Promise.all.

Create those files in bounded batches while keeping the 11,000-file coverage. These errors do not establish that the replacement watcher itself exhausts file descriptors.

Caused by this PR — VSCode directory watching (18 failures)

watch.spec.ts:22, :200, :593, :631, :732, :788 time out across default, default-reuse, and default-trace. The extension passes directories through watch({ fileNames: [...] }), including paths ending in /. TestRunner.watch() now places those paths in FSWatcherParams.files, which only watches their parent and accepts exact file matches. It no longer recursively watches directory inputs. In the initial watch-all flow, no listTests() call has populated the separate dirs set.

I reproduced the difference with the exact base (f9a841f968dd) and PR watcher sources: editing a child of a directory supplied this way reports a change before the PR, but not after it. An explicit file input still works. Normalize and classify directory inputs as recursive dirs rather than individual files.

The refreshed CI results database contains the new test only on this PR and has no VSCode results. Neither group is being classified as a pre-existing flake; the verdict rests on the CI errors and the watcher comparison.

Triaged by the Playwright bot - agent run

@dgozman Dmitry Gozman (dgozman) left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are failures on the watch mode tests, let's fix them.

The VS Code extension passes the test dir when watching all tests.
Also remove the test with 11000 files.
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

23597 passed, 707 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

5194 passed, 616 skipped


Merge workflow run.

@dgozman
Dmitry Gozman (dgozman) marked this pull request as ready for review October 29, 2026 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: UI mode fails with spawn EBADF on macOS when the test directory contains more than ~10k files

2 participants