-
-
Notifications
You must be signed in to change notification settings - Fork 34.8k
module: add clearCache for CJS and ESM #61767
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
1d0accc
b3bd79a
0507308
ba8ffaa
af0f7d0
83d2402
ee06977
fb387a4
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -66,6 +66,58 @@ const require = createRequire(import.meta.url); | |
| const siblingModule = require('./sibling-module'); | ||
| ``` | ||
| ### `module.clearCache(specifier[, options])` | ||
| <!-- YAML | ||
| added: REPLACEME | ||
| --> | ||
| > Stability: 1.1 - Active development | ||
| * `specifier` {string|URL} The module specifier or URL to resolve. The resolved URL/filename | ||
| is cleared from the load cache; the specifier (with `parentURL`) is cleared from the | ||
| resolve cache. | ||
| * `options` {Object} | ||
| * `parentURL` {string|URL} The parent URL used to resolve non-URL specifiers. | ||
| For CommonJS, pass `pathToFileURL(__filename)`. For ES modules, pass `import.meta.url`. | ||
| * Returns: {Object} An object with `{ require: boolean, import: boolean }` indicating whether entries | ||
| were removed from each cache. | ||
| Clears the CommonJS `require` cache and the ESM module cache for a module. This enables | ||
| reload patterns similar to deleting from `require.cache` in CommonJS, and is useful for HMR. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This may need to spell out a bit more concretely how HMR is supposed to use this API without leaking more. From #61767 (comment) there are a couple of things that need to be taken care of:
2 has a subtle difference in CommonJS v.s. ESM dependents. If the changed module has CommonJS dependents, the solution may need to track and clear the cache of all the dependent modules and re-evaluate from the root to see the update. For ESM dependents it's easier because evaluation is separate from linking, so the solution can simply clear the cache for the changed module and the root, and re-evaluate from the root to pick up the new branch while reusing all the existing branches (so unchanged modules won't get evaluated again). This won't violate the spec as explained by Nicolo because it's technically disposing the old graph and then linking and evaluating a new graph again that just reuse some existing module records, not mutating the existing link of an old, evaluated graph. Although, it would be much simpler if whatever that uses it enforces that the module that gets HMR can only ever be the root itself and no bottom-up dependency HMR is supported. But I am not sure if that's a limitation that all existing HMR users can enforce. |
||
| Resolution failures for one module system do not throw; check the returned flags to see what | ||
| was cleared. | ||
| This does not clear resolution cache entries for that specifier. Clearing a module does not | ||
| clear cached entries for its dependencies, and other specifiers that resolve to the same target | ||
| may remain. | ||
| When a `file:` URL is resolved, cached module jobs for the same file path are cleared even if | ||
| they differ by search or hash. | ||
| If the same file is loaded via multiple specifiers (for example `require('./x')` alongside | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this also needs to explain that if both |
||
| `import('./x.js?t=1')` and `import('./x.js?t=2')`), resolution cache entries for each specifier | ||
| remain. Use consistent specifiers, or call `clearCache()` for each specifier you want to | ||
| re-execute. | ||
| ```mjs | ||
| import { clearCache } from 'node:module'; | ||
|
|
||
| const url = new URL('./mod.mjs', import.meta.url); | ||
| await import(url.href); | ||
|
|
||
| clearCache(url); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This still needs examples showing what clearing for non-absolute specifiers would do. |
||
| await import(url.href); // re-executes the module | ||
| ``` | ||
| ```cjs | ||
| const { clearCache } = require('node:module'); | ||
| const path = require('node:path'); | ||
|
|
||
| const file = path.join(__dirname, 'mod.js'); | ||
| require(file); | ||
|
|
||
| clearCache(file); | ||
| require(file); // re-executes the module | ||
| ``` | ||
| ### `module.findPackageJSON(specifier[, base])` | ||
| <!-- YAML | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,330 @@ | ||
| // Copyright Joyent, Inc. and other Node contributors. | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We only keep copyright headers in older files. New files don't need this and are governed by the top-level LICENSE - unless you work is specifically sponsored by Joyent and you specifically want this, of course ;) |
||
| // | ||
| // Permission is hereby granted, free of charge, to any person obtaining a | ||
| // copy of this software and associated documentation files (the | ||
| // "Software"), to deal in the Software without restriction, including | ||
| // without limitation the rights to use, copy, modify, merge, publish, | ||
| // distribute, sublicense, and/or sell copies of the Software, and to permit | ||
| // persons to whom the Software is furnished to do so, subject to the | ||
| // following conditions: | ||
| // | ||
| // The above copyright notice and this permission notice shall be included | ||
| // in all copies or substantial portions of the Software. | ||
| // | ||
| // THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS | ||
| // OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF | ||
| // MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN | ||
| // NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, | ||
| // DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR | ||
| // OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE | ||
| // USE OR OTHER DEALINGS IN THE SOFTWARE. | ||
|
|
||
| 'use strict'; | ||
|
|
||
| const { | ||
| ArrayIsArray, | ||
| ArrayPrototypeIndexOf, | ||
| ArrayPrototypePush, | ||
| ArrayPrototypeSplice, | ||
| ObjectKeys, | ||
| StringPrototypeCharCodeAt, | ||
| StringPrototypeStartsWith, | ||
| } = primordials; | ||
|
|
||
| const { Module, resolveForCJSWithHooks } = require('internal/modules/cjs/loader'); | ||
| const { fileURLToPath, isURL, URLParse, pathToFileURL } = require('internal/url'); | ||
| const { kEmptyObject, isWindows } = require('internal/util'); | ||
| const { validateObject, validateString } = require('internal/validators'); | ||
| const { | ||
| codes: { | ||
| ERR_INVALID_ARG_VALUE, | ||
| }, | ||
| } = require('internal/errors'); | ||
| const { CHAR_DOT } = require('internal/constants'); | ||
| const path = require('path'); | ||
| const { | ||
| privateSymbols: { | ||
| module_first_parent_private_symbol: kFirstModuleParent, | ||
| module_last_parent_private_symbol: kLastModuleParent, | ||
| }, | ||
| } = internalBinding('util'); | ||
|
|
||
| /** | ||
| * Normalize the parent URL for cache clearing. | ||
| * @param {string|URL|undefined} parentURL | ||
| * @returns {{ parentURL: string|undefined, parentPath: string|undefined }} | ||
| */ | ||
| function normalizeClearCacheParent(parentURL) { | ||
| if (parentURL === undefined) { | ||
| return { __proto__: null, parentURL: undefined, parentPath: undefined }; | ||
| } | ||
|
|
||
| if (isURL(parentURL)) { | ||
| let parentPath; | ||
| if (parentURL.protocol === 'file:' && parentURL.search === '' && parentURL.hash === '') { | ||
| parentPath = fileURLToPath(parentURL); | ||
| } | ||
| return { __proto__: null, parentURL: parentURL.href, parentPath }; | ||
| } | ||
|
|
||
| validateString(parentURL, 'options.parentURL'); | ||
| const url = URLParse(parentURL); | ||
| if (!url) { | ||
| throw new ERR_INVALID_ARG_VALUE('options.parentURL', parentURL, | ||
| 'must be a URL'); | ||
| } | ||
|
|
||
| let parentPath; | ||
| if (url.protocol === 'file:' && url.search === '' && url.hash === '') { | ||
| parentPath = fileURLToPath(url); | ||
| } | ||
| return { __proto__: null, parentURL: url.href, parentPath }; | ||
| } | ||
|
|
||
| /** | ||
| * Parse a specifier as a URL when possible. | ||
| * @param {string|URL} specifier | ||
| * @returns {URL|null} | ||
| */ | ||
| function getURLFromClearCacheSpecifier(specifier) { | ||
| if (isURL(specifier)) { | ||
| return specifier; | ||
| } | ||
|
|
||
| if (typeof specifier !== 'string' || path.isAbsolute(specifier)) { | ||
| return null; | ||
| } | ||
|
|
||
| return URLParse(specifier) ?? null; | ||
| } | ||
|
|
||
| /** | ||
| * Create a synthetic parent module for CJS resolution. | ||
| * @param {string} parentPath | ||
| * @returns {Module} | ||
| */ | ||
| function createParentModuleForClearCache(parentPath) { | ||
| const parent = new Module(parentPath); | ||
| parent.filename = parentPath; | ||
| parent.paths = Module._nodeModulePaths(path.dirname(parentPath)); | ||
| return parent; | ||
| } | ||
|
|
||
| /** | ||
| * Resolve a cache filename for CommonJS. | ||
| * @param {string|URL} specifier | ||
| * @param {string|undefined} parentPath | ||
| * @returns {string|null} | ||
| */ | ||
| function resolveClearCacheFilename(specifier, parentPath) { | ||
| if (!parentPath && typeof specifier === 'string' && isRelative(specifier)) { | ||
| return null; | ||
| } | ||
|
|
||
| const parsedURL = getURLFromClearCacheSpecifier(specifier); | ||
| let request = specifier; | ||
| if (parsedURL) { | ||
| if (parsedURL.protocol !== 'file:' || parsedURL.search !== '' || parsedURL.hash !== '') { | ||
| return null; | ||
| } | ||
| request = fileURLToPath(parsedURL); | ||
| } | ||
|
|
||
| const parent = parentPath ? createParentModuleForClearCache(parentPath) : null; | ||
| const { filename, format } = resolveForCJSWithHooks(request, parent, false, false); | ||
|
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This and |
||
| if (format === 'builtin') { | ||
| return null; | ||
| } | ||
| return filename; | ||
| } | ||
|
|
||
| /** | ||
| * Resolve a cache URL for ESM. | ||
| * @param {string|URL} specifier | ||
| * @param {string|undefined} parentURL | ||
| * @returns {string} | ||
| */ | ||
| function resolveClearCacheURL(specifier, parentURL) { | ||
| const parsedURL = getURLFromClearCacheSpecifier(specifier); | ||
| if (parsedURL != null) { | ||
| return parsedURL.href; | ||
| } | ||
|
|
||
| if (path.isAbsolute(specifier)) { | ||
| return pathToFileURL(specifier).href; | ||
| } | ||
|
|
||
| if (parentURL === undefined) { | ||
| throw new ERR_INVALID_ARG_VALUE('options.parentURL', parentURL, | ||
| 'must be provided for non-URL ESM specifiers'); | ||
| } | ||
|
|
||
| const cascadedLoader = | ||
| require('internal/modules/esm/loader').getOrInitializeCascadedLoader(); | ||
| const request = { specifier, __proto__: null }; | ||
| return cascadedLoader.resolveSync(parentURL, request).url; | ||
| } | ||
|
|
||
| /** | ||
| * Remove cached module references from parent children arrays. | ||
| * @param {Module} targetModule | ||
| * @returns {boolean} true if any references were removed. | ||
| */ | ||
| function deleteModuleFromParents(targetModule) { | ||
| const keys = ObjectKeys(Module._cache); | ||
| let deleted = false; | ||
| for (let i = 0; i < keys.length; i++) { | ||
| const cachedModule = Module._cache[keys[i]]; | ||
| if (cachedModule?.[kFirstModuleParent] === targetModule) { | ||
| cachedModule[kFirstModuleParent] = undefined; | ||
| deleted = true; | ||
| } | ||
| if (cachedModule?.[kLastModuleParent] === targetModule) { | ||
| cachedModule[kLastModuleParent] = undefined; | ||
| deleted = true; | ||
| } | ||
| const children = cachedModule?.children; | ||
| if (!ArrayIsArray(children)) { | ||
| continue; | ||
| } | ||
| const index = ArrayPrototypeIndexOf(children, targetModule); | ||
| if (index !== -1) { | ||
| ArrayPrototypeSplice(children, index, 1); | ||
| deleted = true; | ||
| } | ||
| } | ||
| return deleted; | ||
| } | ||
|
|
||
| /** | ||
| * Resolve a file path for a file URL, stripping search/hash. | ||
| * @param {string} url | ||
| * @returns {string|null} | ||
| */ | ||
| function getFilePathFromClearCacheURL(url) { | ||
| const parsedURL = URLParse(url); | ||
| if (parsedURL?.protocol !== 'file:') { | ||
| return null; | ||
| } | ||
|
|
||
| if (parsedURL.search !== '' || parsedURL.hash !== '') { | ||
| parsedURL.search = ''; | ||
| parsedURL.hash = ''; | ||
| } | ||
|
|
||
| try { | ||
| return fileURLToPath(parsedURL); | ||
| } catch { | ||
| return null; | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Remove load cache entries for a URL and its file-path variants. | ||
| * @param {import('internal/modules/esm/module_map').LoadCache} loadCache | ||
| * @param {string} url | ||
| * @returns {boolean} true if any entries were deleted. | ||
| */ | ||
| function deleteLoadCacheEntries(loadCache, url) { | ||
| let deleted = loadCache.deleteAll(url); | ||
| const filename = getFilePathFromClearCacheURL(url); | ||
| if (!filename) { | ||
| return deleted; | ||
| } | ||
|
|
||
| const urls = []; | ||
| for (const entry of loadCache) { | ||
| ArrayPrototypePush(urls, entry[0]); | ||
| } | ||
|
|
||
| for (let i = 0; i < urls.length; i++) { | ||
| const cachedURL = urls[i]; | ||
| if (cachedURL === url) { | ||
| continue; | ||
| } | ||
| const cachedFilename = getFilePathFromClearCacheURL(cachedURL); | ||
| if (cachedFilename === filename) { | ||
| loadCache.deleteAll(cachedURL); | ||
| deleted = true; | ||
| } | ||
| } | ||
|
|
||
| return deleted; | ||
| } | ||
|
|
||
| /** | ||
| * Checks if a path is relative | ||
| * @param {string} pathToCheck the target path | ||
| * @returns {boolean} true if the path is relative, false otherwise | ||
| */ | ||
| function isRelative(pathToCheck) { | ||
| if (StringPrototypeCharCodeAt(pathToCheck, 0) !== CHAR_DOT) { return false; } | ||
|
|
||
| return pathToCheck.length === 1 || pathToCheck === '..' || | ||
| StringPrototypeStartsWith(pathToCheck, './') || | ||
| StringPrototypeStartsWith(pathToCheck, '../') || | ||
| ((isWindows && StringPrototypeStartsWith(pathToCheck, '.\\')) || | ||
| StringPrototypeStartsWith(pathToCheck, '..\\')); | ||
| } | ||
|
|
||
| /** | ||
| * Clear CommonJS and/or ESM module cache entries. | ||
| * @param {string|URL} specifier | ||
| * @param {object} [options] | ||
| * @param {string|URL} [options.parentURL] | ||
| * @returns {{ require: boolean, import: boolean }} | ||
| */ | ||
| function clearCache(specifier, options = kEmptyObject) { | ||
| const isSpecifierURL = isURL(specifier); | ||
| if (!isSpecifierURL) { | ||
| validateString(specifier, 'specifier'); | ||
| } | ||
|
|
||
| validateObject(options, 'options'); | ||
| const { parentURL, parentPath } = normalizeClearCacheParent(options.parentURL); | ||
| const result = { __proto__: null, require: false, import: false }; | ||
|
|
||
| try { | ||
| const deleteCommonjsCachesForFilename = (filename) => { | ||
| let deleted = false; | ||
| const cachedModule = Module._cache[filename]; | ||
| if (cachedModule !== undefined) { | ||
| delete Module._cache[filename]; | ||
| deleted = true; | ||
| deleteModuleFromParents(cachedModule); | ||
| } | ||
| return deleted; | ||
| }; | ||
|
|
||
| const filename = resolveClearCacheFilename(specifier, parentPath); | ||
| if (filename) { | ||
| result.require = deleteCommonjsCachesForFilename(filename); | ||
| } | ||
|
|
||
| if (parentURL !== undefined) { | ||
| const url = resolveClearCacheURL(specifier, parentURL); | ||
| const resolvedPath = getFilePathFromClearCacheURL(url); | ||
| if (resolvedPath && resolvedPath !== filename) { | ||
| if (deleteCommonjsCachesForFilename(resolvedPath)) { | ||
| result.require = true; | ||
| } | ||
| } | ||
| } | ||
|
|
||
| const url = resolveClearCacheURL(specifier, parentURL); | ||
| const cascadedLoader = | ||
| require('internal/modules/esm/loader').getOrInitializeCascadedLoader(); | ||
| const loadDeleted = deleteLoadCacheEntries(cascadedLoader.loadCache, url); | ||
| const { clearCjsCache } = require('internal/modules/esm/translators'); | ||
| const cjsCacheDeleted = clearCjsCache(url); | ||
| result.import = loadDeleted || cjsCacheDeleted; | ||
| } catch { | ||
| // Best effort: avoid throwing for require cache clearing. | ||
| } | ||
|
|
||
| return result; | ||
| } | ||
|
|
||
| module.exports = { | ||
| clearCache, | ||
| }; | ||
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What does this do when you call
module.clearCache('./x.js')from saypath/to/y.jsbut do not pass in parentURL? I think the current API hints that it will resolve topath/to/x.jsbut in reality this actually resolves tocwd/x.js? If that's intentional this needs a test - although I think resolving relative to cwd might be rather surprising.It might be better to just enforce that either it's a full URL, or if it's not, parentURL is mandatory. Or alternatively - never takes specifiers, always take full URLs. Users can get to the URL via
import.meta.resolveorrequire.resolveor with more rudimentary methods like URL/path.join in the examples below. Then they can cache the URL themselves as needed. And the API is only in charge of clearing the load cache entry corresponding to that URL. That'll also simplify the implementation a lot, and it no longer needs to create a fake parent for CJS resolution.