Skip to content

module: cache nearest parent package.json per directory - #65326

Open
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/module-parent-package-json-dir-cache
Open

module: cache nearest parent package.json per directory#65326
codebytere wants to merge 1 commit into
nodejs:mainfrom
codebytere:perf/module-parent-package-json-dir-cache

Conversation

@codebytere

@codebytere codebytere commented Aug 16, 2026

Copy link
Copy Markdown
Member

Cold require() of a 1000-module CommonJS tree gets ~6 % faster (100 runs, ***) and module/module-require.js type='.js'
+13.7 %, by looking up the nearest parent package.json once per directory instead of once per file.

module/module-require.js n=10000 type='.js'                                     ***    13.69 %  ±2.36%
module/module-require.js n=10000 type='.json' | 'dir' (don't use this path)              -2.1 %  (relink noise is ±1 %)
cold require() of a generated 1000-module tree, flat layout          (100 runs) ***     6.37 %  ±1.76%
  same, pnpm-style symlinked node_modules                                       ***     6.56 %  ±0.90%
  same, with NODE_COMPILE_CACHE warm                                            ***     5.10 %  ±1.43%

getNearestParentPackageJSON(checkPath) - called for every CommonJS module to find "type", and by findPackageJSON -
memoized per file, so N modules in a directory meant N calls into the native package.json reader. The result only depends
on the directory the traversal starts from, so the JS-side cache is now keyed by that directory, derived with the same rule
the native side uses (trailing separator → that directory, else dirname). With the permission model enabled the per-path
behavior is kept, because the native call performs a per-path read check. Same return values, same objects, same lifetime
(both caches were already permanent). 1000 native calls → 236 for the tree above.

Tests: behavior is unchanged; test-module-*, test-require-*, test-esm-*, es-module and test-permission-*
(which exercises the per-path branch) pass. Happy to add a call-count test if wanted.


Disclosure: the code, test, measurements and this description were written by Claude Code, directed and reviewed by @codebytere.

getNearestParentPackageJSON() memoized its answer per file, so every
module loaded still made one native call, and TraverseParent() builds
several std::filesystem::path temporaries per directory level and
serializes the whole package.json, which the JS side then usually
discarded because it already had that package.json deserialized.

The native traversal starts at the directory of the given path, so the
answer only depends on that directory: key the memo by it (following
NormalizePath()'s trailing-separator rule), so that all modules in a
directory share one native call. When the permission model is enabled
the traversal also depends on the read permissions in effect at call
time, so that configuration keeps the per-file cache.

Loading a 1000-module tree spread over ~240 directories goes from 1000
to 236 native calls.

Signed-off-by: Shelley Vohr <shelley.vohr@gmail.com>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/loaders

@nodejs-github-bot nodejs-github-bot added module Issues and PRs related to the module subsystem. needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codebytere codebytere added request-ci Add this label to start a Jenkins CI on a PR. and removed needs-ci PRs that need a full CI run. labels Aug 16, 2026
@codecov

codecov Bot commented Aug 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.95918% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.14%. Comparing base (30bff4a) to head (cbba4de).
⚠️ Report is 6 commits behind head on main.

Files with missing lines Patch % Lines
lib/internal/modules/package_json_reader.js 97.95% 1 Missing ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #65326   +/-   ##
=======================================
  Coverage   90.13%   90.14%           
=======================================
  Files         752      752           
  Lines      251568   251599   +31     
  Branches    47270    47289   +19     
=======================================
+ Hits       226759   226799   +40     
+ Misses      16168    16146   -22     
- Partials     8641     8654   +13     
Files with missing lines Coverage Δ
lib/internal/modules/package_json_reader.js 99.23% <97.95%> (-0.22%) ⬇️

... and 32 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Aug 16, 2026
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

module Issues and PRs related to the module subsystem.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants