Skip to content

Conversation

@elrayle
Copy link
Collaborator

@elrayle elrayle commented Jan 26, 2026

Benchmarks were added to test efficiency of several ways to test that a license is a valid SPDX license:

  1. spdxexp.ValidateLicenses([]string{"MIT”})
  2. spdxexp.ActiveLicense("MIT")
  3. ”MIT” == “MIT”

NOTE: The array []string{"MIT”}) is created outside the benchmark loop to avoid it contributing to the benchmark time.

Two benchmarks were added:

1) `spdxexp.ValidateLicenses([]string{"MIT”})`
2) `”MIT” == “MIT”`

NOTE: The array `[]string{"MIT”})` is created outside the benchmark loop to avoid it contributing to the benchmark time.
@elrayle elrayle requested a review from dangoor as a code owner January 26, 2026 15:20
Copilot AI review requested due to automatic review settings January 26, 2026 15:20
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds Go benchmarks to measure and compare the performance and allocation behavior of SPDX license validation for a simple MIT license against a baseline string equality check.

Changes:

  • Added BenchmarkValidateLicensesMIT to benchmark spdxexp.ValidateLicenses([]string{"MIT"}), with the license slice allocated outside the benchmark loop and allocations reported.
  • Added BenchmarkStringEqualityMIT as a control benchmark that measures the cost of a simple "MIT" == "MIT" comparison with allocation reporting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The benchmark and summary will allow us to quantify changes to the algorithm  to help us work to a more efficient and scalable evaluation process.
@elrayle
Copy link
Collaborator Author

elrayle commented Jan 26, 2026

Results:

$ go test ./cmd -run '^$' -bench 'Benchmark(ValidateLicensesMIT|ActiveLicenseMIT|StringEqualityMIT)$' -benchmem -benchtime=1000x -count=10
Benchmark output columns (Go 'go test -bench'):
- BenchmarkName-<GOMAXPROCS>: which benchmark ran and with how many OS threads
- iters: number of iterations (b.N) executed
- ns/op: average time per iteration
- B/op: bytes allocated per iteration (shown with -benchmem)
- allocs/op: allocations per iteration (shown with -benchmem)

goos: darwin
goarch: arm64
pkg: github.com/github/go-spdx/v2/cmd
cpu: Apple M1 Pro
BenchmarkValidateLicensesMIT-10             1000              6014 ns/op           14447 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              4517 ns/op           14443 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              5162 ns/op           14485 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              5475 ns/op           14406 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              4436 ns/op           14443 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              3995 ns/op           14554 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              5499 ns/op           14517 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              4854 ns/op           14480 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              4267 ns/op           14480 B/op         30 allocs/op
BenchmarkValidateLicensesMIT-10             1000              5497 ns/op           14517 B/op         30 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2758 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              3047 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2496 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              3652 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2451 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2758 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2444 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2497 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              3378 ns/op           12288 B/op          1 allocs/op
BenchmarkActiveLicenseMIT-10                1000              2894 ns/op           12288 B/op          1 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.5000 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4580 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.5000 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.5000 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4590 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4580 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.5000 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4580 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4580 ns/op          0 B/op          0 allocs/op
BenchmarkStringEqualityMIT-10               1000                 0.4590 ns/op          0 B/op          0 allocs/op
PASS

Scalability summary (at a glance)
+------------------------+--------------+----------------------+------------------------------+
| Characteristic         | MIT==MIT     | activeLicense("MIT") | ValidateLicenses(["MIT"])    |
+------------------------+--------------+----------------------+------------------------------+
| ns/op average          | ~0.5 ns/op   | ~2,616 ns/op         | ~5,379 ns/op                 |
| Scale                  | 1x           | ~4,800x              | ~9,900x                      |
| Time per check         | O(1)         | ~O(N*L)              | ~O(M*L)                      |
| Memory per check       | O(1)         | ~O(N) bytes          | ~O(M) allocs                 |
+------------------------+--------------+----------------------+------------------------------+

Measurement tip: for strict comparisons, keep ops/run equal (-benchtime=1000x) and increase repeats (-count=10+) then compare with benchstat.
ok      github.com/github/go-spdx/v2/cmd        0.493s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants