-
-
Notifications
You must be signed in to change notification settings - Fork 35.2k
module: add clearCache for CJS and ESM #61767
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
b7788b4
357917f
f7c826e
723300c
2496f25
e2a581d
d21982c
35d207e
105ff7e
7e9a7c5
1410c00
933b744
0b472a6
ddde84e
be1f4a5
3021478
ca82056
d7340c0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,6 +66,141 @@ const require = createRequire(import.meta.url); | |
| const siblingModule = require('./sibling-module'); | ||
| ``` | ||
|
|
||
| ### `module.clearCache(specifier, options)` | ||
|
|
||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
|
|
||
| > Stability: 1.0 - Early development | ||
|
|
||
| * `specifier` {string|URL} The module specifier, as it would have been passed to | ||
| `import()` or `require()`. | ||
| * `options` {Object} | ||
| * `parentURL` {string|URL} The parent URL used to resolve the specifier. Parent identity | ||
| is part of the resolution cache key. For CommonJS, pass `pathToFileURL(__filename)`. | ||
| For ES modules, pass `import.meta.url`. | ||
| * `resolver` {string} Specifies how resolution should be performed. Must be either | ||
| `'import'` or `'require'`. | ||
| * `caches` {string} Specifies which caches to clear. Must be one of: | ||
| * `'resolution'` — only clear the resolution cache entry for this specifier. | ||
| * `'module'` — clear the cached module everywhere in Node.js (not counting | ||
| JS-level references). | ||
| * `'all'` — clear both resolution and module caches. | ||
|
|
||
| Clears module resolution and/or module caches for a module. This enables | ||
| reload patterns similar to deleting from `require.cache` in CommonJS, and is useful for | ||
| hot module reload. | ||
|
|
||
| When `caches` is `'module'` or `'all'`, the specifier is resolved using the chosen `resolver` | ||
| and the resolved module is removed from all internal caches (CommonJS `require` cache, ESM | ||
| load cache, and ESM translators cache). When a `file:` URL is resolved, cached module jobs for | ||
| the same file path are cleared even if they differ by search or hash. This means clearing | ||
| `'./mod.mjs?v=1'` will also clear `'./mod.mjs?v=2'` and any other query/hash variants that | ||
| resolve to the same file. | ||
|
|
||
| When `caches` is `'resolution'` or `'all'` with `resolver` set to `'import'`, all ESM | ||
| resolution cache entries for the given `(specifier, parentURL)` pair are cleared regardless | ||
| of import attributes. When `resolver` is `'require'`, the specific CJS relative resolve | ||
| cache entry for the `(parentDir, specifier)` pair is cleared, together with any cached | ||
| `package.json` data for the resolved module's package. | ||
|
|
||
| Clearing a module does not clear cached entries for its dependencies. When using | ||
| `resolver: 'import'`, resolution cache entries for other specifiers that resolve to the | ||
| same target are not cleared — only entries for the exact `(specifier, parentURL)` pair | ||
| are removed. The module cache itself is cleared by resolved file path, so all specifiers | ||
| pointing to the same file will see a fresh execution on next import. | ||
|
|
||
| #### Memory retention and static imports | ||
|
|
||
| `clearCache` only removes references from the Node.js **JavaScript-level** caches | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is there a possibility we could track this and emit a warning? Specifically, if we know that a module is only held at the javascript-level cache (e.g. dynamic import, etc) vs. if we know it likely would leak then we can emit a warning at the very least when clearing the cache may lead to a leak. Doesn't necessarily need to be done in this PR so consider it a non-blocking suggestion/question.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can take a look but we can tackle this after the initial version has landed. It's marked as "active development". |
||
| (the ESM load cache, resolve cache, CJS `require.cache`, and related structures). | ||
| It does **not** affect V8-internal module graph references. | ||
|
|
||
| When a module M is **statically imported** by a live parent module P | ||
| (i.e., via a top-level `import … from '…'` statement that has already been | ||
| evaluated), V8's module instantiation creates a permanent internal strong | ||
| reference from P's compiled module record to M's module record. Calling | ||
| `clearCache(M)` cannot sever that link. Consequences: | ||
|
|
||
| * The old instance of M **stays alive in memory** for as long as P is alive, | ||
| regardless of how many times M is cleared and re-imported. | ||
| * A fresh `import(M)` after clearing will create a **separate** module instance | ||
| that new importers see. P, however, continues to use the original instance — | ||
| the two coexist simultaneously (sometimes called a "split-brain" state). | ||
| * This is a **bounded** retention: one stale module instance per cleared module | ||
| per live static parent. It does not grow unboundedly across clear/re-import | ||
| cycles. | ||
|
|
||
| For **dynamically imported** modules (`await import('./M.mjs')` with no live | ||
| static parent holding the result), the old `ModuleWrap` becomes eligible for | ||
| garbage collection once `clearCache` removes it from Node.js caches and all | ||
| JS-land references (e.g., stored namespace objects) are dropped. | ||
|
|
||
| The safest pattern for hot-reload of ES modules is to use cache-busting search | ||
| parameters (so each version is a distinct module URL) and use dynamic imports for modules that need to be reloaded: | ||
|
|
||
| #### ECMA-262 spec considerations | ||
|
|
||
| Re-importing the exact same `(specifier, parentURL, importAttributes)` tuple after clearing the module cache | ||
| technically violates the idempotency invariant of the ECMA-262 | ||
| [`HostLoadImportedModule`][] host hook, which expects that the same module request always | ||
| returns the same Module Record for a given referrer. The result of violating this requirement | ||
| is undefined — e.g. it can lead to crashes. For spec-compliant usage, use | ||
| cache-busting search parameters so that each reload uses a distinct module request: | ||
|
|
||
| ```mjs | ||
| import { clearCache } from 'node:module'; | ||
| import { watch } from 'node:fs'; | ||
|
|
||
| let version = 0; | ||
| const base = new URL('./app.mjs', import.meta.url); | ||
|
|
||
| watch(base, async () => { | ||
| // Clear the module cache for the previous version. | ||
| clearCache(new URL(`${base.href}?v=${version}`), { | ||
| parentURL: import.meta.url, | ||
| resolver: 'import', | ||
| caches: 'all', | ||
| }); | ||
| version++; | ||
| // Re-import with a new search parameter — this is a distinct module request | ||
| // and does not violate the ECMA-262 invariant. | ||
| const mod = await import(`${base.href}?v=${version}`); | ||
| console.log('reloaded:', mod); | ||
| }); | ||
| ``` | ||
|
|
||
| #### Examples | ||
|
|
||
| ```mjs | ||
| import { clearCache } from 'node:module'; | ||
|
|
||
| await import('./mod.mjs'); | ||
|
|
||
| clearCache('./mod.mjs', { | ||
| parentURL: import.meta.url, | ||
| resolver: 'import', | ||
| caches: 'module', | ||
| }); | ||
| await import('./mod.mjs'); // re-executes the module | ||
| ``` | ||
|
|
||
| ```cjs | ||
| const { clearCache } = require('node:module'); | ||
| const { pathToFileURL } = require('node:url'); | ||
|
|
||
| require('./mod.js'); | ||
|
|
||
| clearCache('./mod.js', { | ||
| parentURL: pathToFileURL(__filename), | ||
| resolver: 'require', | ||
| caches: 'module', | ||
| }); | ||
| require('./mod.js'); // eslint-disable-line node-core/no-duplicate-requires | ||
| // re-executes the module | ||
| ``` | ||
anonrig marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### `module.findPackageJSON(specifier[, base])` | ||
|
|
||
| <!-- YAML | ||
|
|
@@ -2010,6 +2145,7 @@ returned object contains the following keys: | |
| [`--enable-source-maps`]: cli.md#--enable-source-maps | ||
| [`--import`]: cli.md#--importmodule | ||
| [`--require`]: cli.md#-r---require-module | ||
| [`HostLoadImportedModule`]: https://tc39.es/ecma262/#sec-HostLoadImportedModule | ||
| [`NODE_COMPILE_CACHE=dir`]: cli.md#node_compile_cachedir | ||
| [`NODE_COMPILE_CACHE_PORTABLE=1`]: cli.md#node_compile_cache_portable1 | ||
| [`NODE_DISABLE_COMPILE_CACHE=1`]: cli.md#node_disable_compile_cache1 | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we really need this feature, or could we leave this customization out for now?
To fully understand the resolve cache case - this means we clear the resolution for
specifier, parentURL, attributesonly right? It seems to me like that has to be expected here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I notice this was marked resolved but it's not clear what the answer was?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please take a look at: 3021478
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unfortunately that commit removes
attributesand not thecachesoption this was referring to.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@joyeecheung would you be open to having the
cachesoption be reconsidered for a follow-on? It would be good to hear the justification for this further. To align spec caching with this PR will be some work yet, and it would help if we can align on a simple model to start.Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Did you mean to only have "all" semantics at all times? I think that's fine, resolution-only clearing is mostly for cache-busting resolutions.
To explain why I thought it would be useful to have separate module cahe clearing - while you don't want leaks, you also won't want use-after-frees and practices in vm.Script/vm.Module suggests that it works alright to allow V8 to retain the references, because once Node.js release them the rest are only retained through user code, when user module graph goes away it will go away, and libraries have been managing that alright. If you try to clear it while user code still rely on old modules you get use after frees and people will be annoyed. With cache busting solutions the library could use some help avoiding clearing the module cache too early even when the no longer allow new resolutions to old modules - and if they can't clear them separately they might be forced into a "leak-or-use-after-free" choice (even though resolution leak is usually minor compared to module leaks, so it's natural to choose resolution leak over module use-after-free which can be a lot more annoying)