feat(bedrock): enable prompt caching toggle for custom ARN and default to ON #10697
+5
−2
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Related GitHub Issue
Closes: #10576
Description
This PR attempts to address Issue #10576. Feedback and guidance are welcome.
Changes:
Enable prompt caching toggle for custom ARN users: Changed
supportsPromptCachefromfalsetotruefor custom ARN model info. Custom ARN is treated as an advanced option where users know what they are doing, and their custom ARN often points to newer Claude models that support prompt caching.Default prompt caching to ON: Changed the default value of
awsUsePromptCachefromfalsetotrueby using?? trueinstead of|| false. This ensures users get cost savings and latency improvements by default, while the nullish coalescing operator (??) properly respects explicitly setfalsevalues.Files modified:
webview-ui/src/components/ui/hooks/useSelectedModel.ts- Enable supportsPromptCache for custom-arnwebview-ui/src/components/settings/providers/Bedrock.tsx- Default checkbox to trueTest Procedure
cd src && npx vitest run api/providers/__tests__/bedrock*- all 143 tests passnpx tsc --noEmit- no type errorsPre-Submission Checklist
Documentation Updates
Additional Notes
This change treats custom ARN as an advanced option where users know what they are doing. Since custom ARNs often point to newer Claude models that support prompt caching, it makes sense to show the toggle and let users decide. This is a re-implementation of the closed PR #10577 per request in the issue comments.
Important
Enable prompt caching for custom ARN users and set default prompt caching to ON in
useSelectedModel.tsandBedrock.tsx.supportsPromptCachetotrueingetSelectedModel()inuseSelectedModel.ts.awsUsePromptCachetotrueusing?? trueinBedrock.tsxto ensure cost savings and latency improvements.useSelectedModel.ts: ModifygetSelectedModel()to enable prompt caching for custom ARN.Bedrock.tsx: Change default checkbox state for prompt caching to true.This description was created by
for 58540ea. You can customize this summary. It will automatically update as commits are pushed.