Skip to content

test: prefer expected failures to skips in WPT status files - #65311

Open
avivkeller wants to merge 1 commit into
nodejs:mainfrom
avivkeller:avoid-skipping-wpt
Open

test: prefer expected failures to skips in WPT status files#65311
avivkeller wants to merge 1 commit into
nodejs:mainfrom
avivkeller:avoid-skipping-wpt

Conversation

@avivkeller

@avivkeller avivkeller commented Aug 15, 2026

Copy link
Copy Markdown
Member

As titled. By avoiding skiping web platform tests, and instead expecting failures, we can better detect when our WPT compatibility changes. For instance, in this PR, several previously skipped WPT files actually pass, and are now run

cc @panva

Signed-off-by: avivkeller <me@aviv.sh>
@nodejs-github-bot

Copy link
Copy Markdown
Collaborator

Review requested:

  • @nodejs/web-standards

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Aug 15, 2026
@avivkeller avivkeller added the wip Issues and PRs that are still a work in progress. label Aug 15, 2026
@codecov

codecov Bot commented Aug 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.13%. Comparing base (30bff4a) to head (3971a94).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main   #65311      +/-   ##
==========================================
- Coverage   90.13%   90.13%   -0.01%     
==========================================
  Files         752      752              
  Lines      251568   251568              
  Branches    47270    47286      +16     
==========================================
- Hits       226759   226757       -2     
+ Misses      16168    16154      -14     
- Partials     8641     8657      +16     

see 35 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@panva

panva commented Aug 16, 2026

Copy link
Copy Markdown
Member

I don't see the point in unskipping

  • tentatives that aren't part of the spec and are known to fail
  • ones that genuinely don't make sense in node, e.g. secure contexts
  • where we simply don't have the API - e.g. FileReader
  • where a WPT server is needed

In general I trust the maintainers of those systems to know better and to curate their WPT expectations and wouldn't cause this churn which is also a PITA when backporting.

@avivkeller

Copy link
Copy Markdown
Member Author

I'm happy to close, I figured this would be easier for the respective maintainers to identify unexpected successes or failures instead of skips

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. wip Issues and PRs that are still a work in progress.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants