Restore per-element tool attribution in tsp50 scorer#1
Open
wittjeff wants to merge 1 commit intojrpool:mainfrom
Open
Restore per-element tool attribution in tsp50 scorer#1wittjeff wants to merge 1 commit intojrpool:mainfrom
wittjeff wants to merge 1 commit intojrpool:mainfrom
Conversation
The tsp50 scorer simplified element tracking from tsp's per-element tool attribution (issuePaths[issueID][pathID] = Set<tools>) to a flat set (issuePaths[issueName] = Set<pathIDs>). This lost the ability for consumers to know which tools detected each element. This change: 1. Restores tool-per-element tracking: issuePaths maps each pathID to the set of tools that reported it, and details.element groups paths by their detecting tool combinations (e.g., "axe + ibm": [paths]). 2. Adds uniqueElementCount to each issue's details: the count of unique path-identified elements, giving consumers a deduplicated instance count based on pathID. This is more accurate than summing per-tool instanceCounts, which double-counts elements found by multiple tools. The element detail format now matches tsp's output structure, making it consistent across scorer versions.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tsp50.jsthat was lost when simplifying fromtsp.jsuniqueElementCountto issue details for accurate instance countingdetails.elemententries by detecting tool combination (matchingtsp.jsbehavior)Problem
tsp50.jstracksissuePathsas a flatSet<pathID>, which loses information about which tools detected each element. The newertsp.jsscorer correctly tracks this as{pathID: Set<toolName>}, enabling downstream consumers to:uniqueElementCount)details.elementWithout this, consumers that rely on
details.elementto understand cross-tool agreement at the element level get only a flat list of paths with no tool attribution.Changes
procs/score/tsp50.js:issuePathsstructure: Changed fromSet<pathID>to{pathID: Set<toolName>}— tracks which tools reported each elementuniqueElementCount: Added to each issue's details — count of unique path-identified elementsdetails.element: Changed from flat array to tool-attributed object — groups paths by detecting tool combination (e.g.,{"axe + ibm": ["/html/body/div[1]"], "axe": ["/html/body/div[2]"]})These changes align
tsp50.jswith the existing behavior intsp.js.Test plan
uniqueElementCountper issuedetails.element[issueID]is grouped by tool combination string🤖 Generated with Claude Code