From 036ee9c8c2e8926553ad02dd8b815f45eca77bab Mon Sep 17 00:00:00 2001 From: sangwook Date: Thu, 29 Jan 2026 11:23:38 +0900 Subject: [PATCH 1/3] feat: add expectFailure enhancements proposal --- proposals/expect-failure-enhancements.md | 64 ++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 proposals/expect-failure-enhancements.md diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md new file mode 100644 index 0000000..66ba1d9 --- /dev/null +++ b/proposals/expect-failure-enhancements.md @@ -0,0 +1,64 @@ +# Feature proposal: `expectFailure` enhancements + +## Summary +Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This aligns with `skip` and `todo` options while adding capabilities similar to `assert.throws`. + +## API & Behavior + +The behavior of `expectFailure` is determined by the type of value provided: + +### 1. String: Failure Reason +When a **non-empty string** is provided, it acts as a documentation message (reason), identical to `skip` and `todo` options. + +```js +test('fails with a specific reason', { + expectFailure: 'Bug #123: Feature not implemented yet' +}, () => { + throw new Error('boom'); +}); +``` +- **Behavior**: The test is expected to fail. The string is treated as a label/reason. +- **Validation**: None. It accepts *any* error. +- **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). +- **Rationale**: Maintains consistency with existing `test` options where a string implies a reason/description. + +### 2. RegExp: Error Matcher +When a **RegExp** is provided, it acts as a validator for the thrown error. + +```js +test('fails with matching error', { + expectFailure: /expected error message/ +}, () => { + throw new Error('this is the expected error message'); +}); +``` +- **Behavior**: The test passes **only if** the thrown error matches the regular expression. +- **Validation**: Strict matching against the error message. +- **Output**: Standard expected failure output. + +## Ambiguity Resolution +Potential ambiguity between "String as Reason" and "String as Matcher" is resolved by strict type separation: +* `typeof value === 'string'` → **Reason** (Documentation only) +* `value instanceof RegExp` → **Matcher** (Validation) + +Users needing to match a specific string error message should use a RegExp (e.g., `/Error message/`) to avoid confusion. + +## Edge Cases & Implementation Details + +### Empty String (`expectFailure: ''`) +Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. + +* `expectFailure: ''` behaves exactly like `expectFailure: false`. +* The feature is **disabled**, and the test is expected to pass normally. + +### Type Safety for `this.passed` +The implementation must ensure that `this.passed` remains a strict `boolean`. +Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution (assigning `""` or `"reason"` instead of `false`/`true`). + +**Recommended Implementation Logic:** +```javascript +// When an error is caught: +this.passed = !!this.expectFailure; // Forces conversion to boolean +``` +* If `expectFailure` is `"reason"` → `true` (Test Passes) +* If `expectFailure` is `""` → `false` (Test Fails, as expected failure was not active) From 049c2d3e83bbb675bdbf45cf8cc8a56db8e6d562 Mon Sep 17 00:00:00 2001 From: sangwook Date: Thu, 29 Jan 2026 13:40:11 +0900 Subject: [PATCH 2/3] docs: add object support for combined reason and validation (fixes #61570 requirement) --- proposals/expect-failure-enhancements.md | 36 +++++++++++++++++------- 1 file changed, 26 insertions(+), 10 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 66ba1d9..7993d08 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -1,11 +1,11 @@ # Feature proposal: `expectFailure` enhancements ## Summary -Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This aligns with `skip` and `todo` options while adding capabilities similar to `assert.throws`. +Update the `expectFailure` option in `test()` to accept different types of values, enabling both **custom failure messages** and **error validation**. This proposal integrates the requirements from [nodejs/node#61570](https://github.com/nodejs/node/issues/61570), ensuring consistency with `skip`/`todo` while adding robust validation capabilities. ## API & Behavior -The behavior of `expectFailure` is determined by the type of value provided: +The behavior of `expectFailure` is strictly determined by the type of value provided: ### 1. String: Failure Reason When a **non-empty string** is provided, it acts as a documentation message (reason), identical to `skip` and `todo` options. @@ -20,7 +20,6 @@ test('fails with a specific reason', { - **Behavior**: The test is expected to fail. The string is treated as a label/reason. - **Validation**: None. It accepts *any* error. - **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). -- **Rationale**: Maintains consistency with existing `test` options where a string implies a reason/description. ### 2. RegExp: Error Matcher When a **RegExp** is provided, it acts as a validator for the thrown error. @@ -36,24 +35,41 @@ test('fails with matching error', { - **Validation**: Strict matching against the error message. - **Output**: Standard expected failure output. -## Ambiguity Resolution -Potential ambiguity between "String as Reason" and "String as Matcher" is resolved by strict type separation: -* `typeof value === 'string'` → **Reason** (Documentation only) -* `value instanceof RegExp` → **Matcher** (Validation) +### 3. Object: Reason & Validation +When an **Object** is provided, it allows specifying both a failure reason and validation logic simultaneously. + +```js +test('fails with reason and specific error', { + expectFailure: { + message: 'Bug #123: Edge case behavior', // Reason + match: /Index out of bounds/ // Validation + } +}, () => { + throw new RangeError('Index out of bounds'); +}); +``` +- **Properties**: + - `message` (String): The failure reason/label (displayed in reporter). + - `match` (RegExp | Object | Function): Validation logic (similar to `assert.throws` validation argument). +- **Behavior**: The test passes **only if** the error matches the `match` criteria. +- **Output**: The reporter displays the `message`. -Users needing to match a specific string error message should use a RegExp (e.g., `/Error message/`) to avoid confusion. +## Ambiguity Resolution +Potential ambiguity is resolved by strict type separation: +* `typeof value === 'string'` → **Reason** +* `value instanceof RegExp` → **Matcher** +* `typeof value === 'object'` → **Both** (Explicit properties) ## Edge Cases & Implementation Details ### Empty String (`expectFailure: ''`) Following standard JavaScript truthiness rules, an empty string should be treated as **falsy**. - * `expectFailure: ''` behaves exactly like `expectFailure: false`. * The feature is **disabled**, and the test is expected to pass normally. ### Type Safety for `this.passed` The implementation must ensure that `this.passed` remains a strict `boolean`. -Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution (assigning `""` or `"reason"` instead of `false`/`true`). +Assigning a string directly (e.g., `this.passed = this.expectFailure`) is unsafe as it introduces type pollution. **Recommended Implementation Logic:** ```javascript From abb5912aa29d0de14e495ca0d79de1a277c661a2 Mon Sep 17 00:00:00 2001 From: sangwook Date: Fri, 30 Jan 2026 00:01:18 +0900 Subject: [PATCH 3/3] docs: update expectFailure proposal with validation details Update the `expectFailure` enhancements proposal based on feedback and implementation alignment: - Consolidate validation logic under the `with` property within an object. - Remove direct RegExp support in favor of the object syntax for consistency. - Specify usage of `assert.throws` for robust error validation. - Document alternatives considered (flat options). --- proposals/expect-failure-enhancements.md | 38 +++++++++++++++++------- 1 file changed, 27 insertions(+), 11 deletions(-) diff --git a/proposals/expect-failure-enhancements.md b/proposals/expect-failure-enhancements.md index 7993d08..122522b 100644 --- a/proposals/expect-failure-enhancements.md +++ b/proposals/expect-failure-enhancements.md @@ -21,19 +21,16 @@ test('fails with a specific reason', { - **Validation**: None. It accepts *any* error. - **Output**: The reporter displays the string (e.g., `# EXPECTED FAILURE Bug #123...`). -### 2. RegExp: Error Matcher -When a **RegExp** is provided, it acts as a validator for the thrown error. +### 2. RegExp: Error Matcher (via Object) +Use the object form with the `with` property. ```js test('fails with matching error', { - expectFailure: /expected error message/ + expectFailure: { with: /expected error message/ } }, () => { throw new Error('this is the expected error message'); }); ``` -- **Behavior**: The test passes **only if** the thrown error matches the regular expression. -- **Validation**: Strict matching against the error message. -- **Output**: Standard expected failure output. ### 3. Object: Reason & Validation When an **Object** is provided, it allows specifying both a failure reason and validation logic simultaneously. @@ -42,7 +39,7 @@ When an **Object** is provided, it allows specifying both a failure reason and v test('fails with reason and specific error', { expectFailure: { message: 'Bug #123: Edge case behavior', // Reason - match: /Index out of bounds/ // Validation + with: /Index out of bounds/ // Validation } }, () => { throw new RangeError('Index out of bounds'); @@ -50,15 +47,34 @@ test('fails with reason and specific error', { ``` - **Properties**: - `message` (String): The failure reason/label (displayed in reporter). - - `match` (RegExp | Object | Function): Validation logic (similar to `assert.throws` validation argument). -- **Behavior**: The test passes **only if** the error matches the `match` criteria. + - `with` (RegExp | Object | Function | Class): Validation logic. This is passed directly to `assert.throws` validation argument, supporting all its capabilities. +- **Behavior**: The test passes **only if** the error matches the `with` criteria. - **Output**: The reporter displays the `message`. ## Ambiguity Resolution Potential ambiguity is resolved by strict type separation: * `typeof value === 'string'` → **Reason** -* `value instanceof RegExp` → **Matcher** -* `typeof value === 'object'` → **Both** (Explicit properties) +* `typeof value === 'object'` → **Configuration Object** (`message` and/or `with`) + +## Alternatives Considered + +### Flat Options (`expectFailureError`) +It was proposed to split the options into `expectFailure` (reason) and `expectFailureError` (validation). +```js +{ + expectFailure: 'reason', + expectFailureError: /error/ +} +``` +This was rejected in favor of the nested object structure to: +1. Keep related configuration grouped. +2. Avoid polluting the top-level options namespace. +3. Allow future extensibility within the `expectFailure` object. + +## Implementation Details + +### Validation Logic +The implementation leverages `assert.throws` internally to perform error validation. This ensures consistency with the existing assertion ecosystem and supports advanced validation (Classes, Custom Functions) out of the box without code duplication. ## Edge Cases & Implementation Details