Fix AWS API Gateway endpoints correlation HTTP span tags - Inferred Proxy Spans#10561
Fix AWS API Gateway endpoints correlation HTTP span tags - Inferred Proxy Spans#10561
Conversation
7163ef9 to
85a368c
Compare
c681b6c to
c8a926f
Compare
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 59 metrics, 12 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.061 s) : 0, 1061384
Total [baseline] (8.759 s) : 0, 8758790
Agent [candidate] (1.075 s) : 0, 1074599
Total [candidate] (8.769 s) : 0, 8769257
section iast
Agent [baseline] (1.238 s) : 0, 1238145
Total [baseline] (9.366 s) : 0, 9365556
Agent [candidate] (1.227 s) : 0, 1226914
Total [candidate] (9.369 s) : 0, 9369409
gantt
title insecure-bank - break down per module: candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.197 ms) : 0, 1197
BytebuddyAgent [baseline] (626.494 ms) : 0, 626494
BytebuddyAgent [candidate] (632.857 ms) : 0, 632857
AgentMeter [baseline] (29.048 ms) : 0, 29048
AgentMeter [candidate] (29.536 ms) : 0, 29536
GlobalTracer [baseline] (256.837 ms) : 0, 256837
GlobalTracer [candidate] (260.774 ms) : 0, 260774
AppSec [baseline] (32.944 ms) : 0, 32944
AppSec [candidate] (33.305 ms) : 0, 33305
Debugger [baseline] (63.077 ms) : 0, 63077
Debugger [candidate] (63.156 ms) : 0, 63156
Remote Config [baseline] (606.266 µs) : 0, 606
Remote Config [candidate] (631.313 µs) : 0, 631
Telemetry [baseline] (9.785 ms) : 0, 9785
Telemetry [candidate] (9.927 ms) : 0, 9927
Flare Poller [baseline] (5.366 ms) : 0, 5366
Flare Poller [candidate] (6.891 ms) : 0, 6891
section iast
crashtracking [baseline] (1.203 ms) : 0, 1203
crashtracking [candidate] (1.185 ms) : 0, 1185
BytebuddyAgent [baseline] (800.82 ms) : 0, 800820
BytebuddyAgent [candidate] (792.995 ms) : 0, 792995
AgentMeter [baseline] (11.551 ms) : 0, 11551
AgentMeter [candidate] (11.263 ms) : 0, 11263
GlobalTracer [baseline] (249.166 ms) : 0, 249166
GlobalTracer [candidate] (247.137 ms) : 0, 247137
IAST [baseline] (27.253 ms) : 0, 27253
IAST [candidate] (26.967 ms) : 0, 26967
AppSec [baseline] (31.506 ms) : 0, 31506
AppSec [candidate] (31.295 ms) : 0, 31295
Debugger [baseline] (68.149 ms) : 0, 68149
Debugger [candidate] (67.669 ms) : 0, 67669
Remote Config [baseline] (527.402 µs) : 0, 527
Remote Config [candidate] (536.762 µs) : 0, 537
Telemetry [baseline] (8.502 ms) : 0, 8502
Telemetry [candidate] (8.563 ms) : 0, 8563
Flare Poller [baseline] (3.418 ms) : 0, 3418
Flare Poller [candidate] (3.394 ms) : 0, 3394
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066150
Total [baseline] (10.881 s) : 0, 10880911
Agent [candidate] (1.073 s) : 0, 1072712
Total [candidate] (10.794 s) : 0, 10793566
section appsec
Agent [baseline] (1.238 s) : 0, 1238442
Total [baseline] (10.958 s) : 0, 10958327
Agent [candidate] (1.247 s) : 0, 1247156
Total [candidate] (11.028 s) : 0, 11027704
section iast
Agent [baseline] (1.236 s) : 0, 1236216
Total [baseline] (11.212 s) : 0, 11212071
Agent [candidate] (1.233 s) : 0, 1232672
Total [candidate] (11.181 s) : 0, 11180601
section profiling
Agent [baseline] (1.188 s) : 0, 1188424
Total [baseline] (10.906 s) : 0, 10906218
Agent [candidate] (1.197 s) : 0, 1196995
Total [candidate] (10.959 s) : 0, 10959378
gantt
title petclinic - break down per module: candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.195 ms) : 0, 1195
crashtracking [candidate] (1.214 ms) : 0, 1214
BytebuddyAgent [baseline] (627.93 ms) : 0, 627930
BytebuddyAgent [candidate] (632.519 ms) : 0, 632519
AgentMeter [baseline] (29.072 ms) : 0, 29072
AgentMeter [candidate] (29.287 ms) : 0, 29287
GlobalTracer [baseline] (257.691 ms) : 0, 257691
GlobalTracer [candidate] (258.881 ms) : 0, 258881
AppSec [baseline] (33.063 ms) : 0, 33063
AppSec [candidate] (33.23 ms) : 0, 33230
Debugger [baseline] (65.473 ms) : 0, 65473
Debugger [candidate] (65.521 ms) : 0, 65521
Remote Config [baseline] (618.622 µs) : 0, 619
Remote Config [candidate] (604.264 µs) : 0, 604
Telemetry [baseline] (9.204 ms) : 0, 9204
Telemetry [candidate] (11.369 ms) : 0, 11369
Flare Poller [baseline] (5.919 ms) : 0, 5919
Flare Poller [candidate] (3.79 ms) : 0, 3790
section appsec
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.198 ms) : 0, 1198
BytebuddyAgent [baseline] (658.498 ms) : 0, 658498
BytebuddyAgent [candidate] (663.547 ms) : 0, 663547
AgentMeter [baseline] (11.963 ms) : 0, 11963
AgentMeter [candidate] (12.034 ms) : 0, 12034
GlobalTracer [baseline] (257.785 ms) : 0, 257785
GlobalTracer [candidate] (259.917 ms) : 0, 259917
IAST [baseline] (25.266 ms) : 0, 25266
IAST [candidate] (25.648 ms) : 0, 25648
AppSec [baseline] (167.472 ms) : 0, 167472
AppSec [candidate] (168.485 ms) : 0, 168485
Debugger [baseline] (66.524 ms) : 0, 66524
Debugger [candidate] (66.573 ms) : 0, 66573
Remote Config [baseline] (661.93 µs) : 0, 662
Remote Config [candidate] (649.493 µs) : 0, 649
Telemetry [baseline] (9.484 ms) : 0, 9484
Telemetry [candidate] (9.338 ms) : 0, 9338
Flare Poller [baseline] (3.68 ms) : 0, 3680
Flare Poller [candidate] (3.723 ms) : 0, 3723
section iast
crashtracking [baseline] (1.219 ms) : 0, 1219
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (800.095 ms) : 0, 800095
BytebuddyAgent [candidate] (796.24 ms) : 0, 796240
AgentMeter [baseline] (11.297 ms) : 0, 11297
AgentMeter [candidate] (11.335 ms) : 0, 11335
GlobalTracer [baseline] (247.559 ms) : 0, 247559
GlobalTracer [candidate] (248.055 ms) : 0, 248055
IAST [baseline] (26.892 ms) : 0, 26892
IAST [candidate] (27.048 ms) : 0, 27048
AppSec [baseline] (33.767 ms) : 0, 33767
AppSec [candidate] (34.6 ms) : 0, 34600
Debugger [baseline] (66.713 ms) : 0, 66713
Debugger [candidate] (65.891 ms) : 0, 65891
Remote Config [baseline] (535.7 µs) : 0, 536
Remote Config [candidate] (535.014 µs) : 0, 535
Telemetry [baseline] (8.671 ms) : 0, 8671
Telemetry [candidate] (8.575 ms) : 0, 8575
Flare Poller [baseline] (3.484 ms) : 0, 3484
Flare Poller [candidate] (3.424 ms) : 0, 3424
section profiling
crashtracking [baseline] (1.182 ms) : 0, 1182
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (680.334 ms) : 0, 680334
BytebuddyAgent [candidate] (684.483 ms) : 0, 684483
AgentMeter [baseline] (8.484 ms) : 0, 8484
AgentMeter [candidate] (8.63 ms) : 0, 8630
GlobalTracer [baseline] (215.301 ms) : 0, 215301
GlobalTracer [candidate] (217.897 ms) : 0, 217897
AppSec [baseline] (32.506 ms) : 0, 32506
AppSec [candidate] (33.108 ms) : 0, 33108
Debugger [baseline] (67.124 ms) : 0, 67124
Debugger [candidate] (66.993 ms) : 0, 66993
Remote Config [baseline] (626.391 µs) : 0, 626
Remote Config [candidate] (637.25 µs) : 0, 637
Telemetry [baseline] (9.104 ms) : 0, 9104
Telemetry [candidate] (9.039 ms) : 0, 9039
Flare Poller [baseline] (3.809 ms) : 0, 3809
Flare Poller [candidate] (4.602 ms) : 0, 4602
ProfilingAgent [baseline] (99.516 ms) : 0, 99516
ProfilingAgent [candidate] (99.824 ms) : 0, 99824
Profiling [baseline] (100.087 ms) : 0, 100087
Profiling [candidate] (100.405 ms) : 0, 100405
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 0 performance regressions! Performance is the same for 18 metrics, 16 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section baseline
no_agent (17.152 ms) : 16981, 17324
. : milestone, 17152,
appsec (18.722 ms) : 18530, 18914
. : milestone, 18722,
code_origins (17.725 ms) : 17550, 17901
. : milestone, 17725,
iast (17.676 ms) : 17496, 17855
. : milestone, 17676,
profiling (21.42 ms) : 21203, 21637
. : milestone, 21420,
tracing (17.649 ms) : 17474, 17824
. : milestone, 17649,
section candidate
no_agent (18.048 ms) : 17864, 18232
. : milestone, 18048,
appsec (18.398 ms) : 18214, 18581
. : milestone, 18398,
code_origins (17.662 ms) : 17489, 17835
. : milestone, 17662,
iast (17.512 ms) : 17338, 17687
. : milestone, 17512,
profiling (18.543 ms) : 18358, 18728
. : milestone, 18543,
tracing (17.813 ms) : 17638, 17989
. : milestone, 17813,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section baseline
no_agent (1.186 ms) : 1175, 1197
. : milestone, 1186,
iast (3.179 ms) : 3138, 3220
. : milestone, 3179,
iast_FULL (5.88 ms) : 5821, 5939
. : milestone, 5880,
iast_GLOBAL (3.634 ms) : 3570, 3699
. : milestone, 3634,
profiling (2.001 ms) : 1984, 2017
. : milestone, 2001,
tracing (1.811 ms) : 1795, 1827
. : milestone, 1811,
section candidate
no_agent (1.27 ms) : 1257, 1284
. : milestone, 1270,
iast (3.217 ms) : 3175, 3259
. : milestone, 3217,
iast_FULL (5.777 ms) : 5719, 5836
. : milestone, 5777,
iast_GLOBAL (3.491 ms) : 3443, 3539
. : milestone, 3491,
profiling (2.016 ms) : 1998, 2033
. : milestone, 2016,
tracing (1.745 ms) : 1731, 1759
. : milestone, 1745,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section baseline
no_agent (15.688 s) : 15688000, 15688000
. : milestone, 15688000,
appsec (14.995 s) : 14995000, 14995000
. : milestone, 14995000,
iast (18.345 s) : 18345000, 18345000
. : milestone, 18345000,
iast_GLOBAL (17.785 s) : 17785000, 17785000
. : milestone, 17785000,
profiling (14.718 s) : 14718000, 14718000
. : milestone, 14718000,
tracing (14.697 s) : 14697000, 14697000
. : milestone, 14697000,
section candidate
no_agent (15.539 s) : 15539000, 15539000
. : milestone, 15539000,
appsec (14.537 s) : 14537000, 14537000
. : milestone, 14537000,
iast (17.987 s) : 17987000, 17987000
. : milestone, 17987000,
iast_GLOBAL (17.775 s) : 17775000, 17775000
. : milestone, 17775000,
profiling (14.928 s) : 14928000, 14928000
. : milestone, 14928000,
tracing (14.607 s) : 14607000, 14607000
. : milestone, 14607000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.60.0-SNAPSHOT~68d3216ff3, baseline=1.60.0-SNAPSHOT~441b1c6e27
dateFormat X
axisFormat %s
section baseline
no_agent (1.481 ms) : 1470, 1493
. : milestone, 1481,
appsec (3.812 ms) : 3589, 4036
. : milestone, 3812,
iast (2.261 ms) : 2191, 2331
. : milestone, 2261,
iast_GLOBAL (2.314 ms) : 2244, 2385
. : milestone, 2314,
profiling (2.093 ms) : 2038, 2149
. : milestone, 2093,
tracing (2.069 ms) : 2016, 2123
. : milestone, 2069,
section candidate
no_agent (1.476 ms) : 1464, 1487
. : milestone, 1476,
appsec (3.774 ms) : 3552, 3996
. : milestone, 3774,
iast (2.263 ms) : 2193, 2333
. : milestone, 2263,
iast_GLOBAL (2.301 ms) : 2231, 2371
. : milestone, 2301,
profiling (2.128 ms) : 2070, 2186
. : milestone, 2128,
tracing (2.066 ms) : 2012, 2120
. : milestone, 2066,
|
35abdf1 to
9e4cfe6
Compare
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 06ba1cf584
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
What Does This Do
This PR implements standardized tags for inferred proxy spans produced by the Java tracer when instrumenting AWS API Gateway (v1 REST and v2 HTTP APIs). The changes align proxy spans with the cross-platform contract defined in RFC-1081 for endpoint discovery and correlation.
Mandatory tags implemented:
aws.httpapi(v2 HTTP API) in addition toaws.apigateway(v1 REST API)serverfor all proxy spansweb, maintained for consistencyhttps://scheme (prevents backend parsing issues)x-dd-proxy-resource-pathheader (resource template path)<Method> <Route>when route available, fallback to<Method> <Path>_dd.appsec.enabledmetric and_dd.appsec.jsontag from root span to proxy span in distributed tracing scenariosOptional tags implemented:
x-dd-proxy-account-idheaderx-dd-proxy-api-idheaderx-dd-proxy-regionheaderarn:aws:apigateway:{region}::/restapis|apis/{api-id}Motivation
This implementation is required by RFC-1081: Endpoint Discovery & Correlation from Inferred Spans
This PR covers the Inferred Proxy Spans portion of the RFC. The Inferred Lambda Spans portion will be addressed in a separate PR #10576.
Additional Notes
ST checked locally
aws_user exclusion: The optional
aws_usertag was intentionally excluded per RFC guidance due to PII concerns (assumed-role session names may contain user identifiers). Implementation requires explicit approval.AppSec tag propagation fix: During implementation, we discovered and fixed a critical issue with AppSec event propagation when inferred proxy spans are present:
being added to spans entirely.
inferred proxy span becomes the local root, child service-entry spans would skip the callback.
inferred proxy parent, enabling proper AppSec event detection and propagation.
This is achieved by adding tags directly to the current span (span.setTag()) in addition to the root span (traceSeg.setTagTop()), ensuring proper event correlation across
distributed traces.
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: APPSEC-61198
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.