* feat: add Claude Opus 4.8 support
* fix: omit Vertex Opus sampling overrides
* fix: preserve Opus adaptive thinking levels
* fix: clamp Anthropic max effort support
* fix: use sha256 for QA mock call ids
* fix: type Anthropic transport test model metadata
* test: update PDF model default for Opus 4.8
* fix(amazon-bedrock): add known model context windows to discovery
Bedrock's ListFoundationModels API does not expose token limits. Discovery
was hardcoding contextWindow: 32000 for every model, causing Claude (1M),
Nova (300K), and other models to hit premature 'Context limit exceeded'
errors and unnecessary session resets.
Adds a lookup table of known context windows for Bedrock models:
- Anthropic Claude: 200K-1M
- Amazon Nova: 128K-1M
- Meta Llama: 128K
- Mistral: 32K-128K
- DeepSeek: 128K
- Cohere: 128K
- AI21 Jamba: 256K
Inference profile prefixes (us., eu., ap., global.) are stripped before
lookup, so us.anthropic.claude-opus-4-6-v1 correctly resolves to 1M.
Also raises the default fallback from 32K to 128K for unknown models —
most modern models have at least 128K context.
Single file change, no type system modifications.
Complementary to #65030 (provenance flag for warning on unknown models).
Fixes#64919
Related: #64250
* add KNOWN_MAX_TOKENS map and expand model coverage
- Add KNOWN_MAX_TOKENS lookup table with Bedrock-optimized values that
balance response quality against quota burndown (5x rate for Claude 3.7+)
- Add missing models to KNOWN_CONTEXT_WINDOWS: Opus 4.7 (1M), Opus 4.1/4.5,
Sonnet 4, Claude 3/3.5 Haiku, DeepSeek V3/V3.2, Google Gemma 3
- Refactor prefix-stripping into shared resolveKnownValue() helper
- Fix: use !== undefined instead of truthy check for table lookups
- Wire resolveKnownMaxTokens into toModelDefinition and resolveInferenceProfiles
Quota burndown context: Bedrock reserves input_tokens + max_tokens from
TPM at request start. For Claude 3.7+, output burns at 5x. The values
in KNOWN_MAX_TOKENS are intentionally conservative (8-16K for Claude)
to maximize concurrent throughput while still allowing useful responses.
Thinking budget is added separately by the runtime.
* remove KNOWN_MAX_TOKENS — maxTokens should be handled upstream
Remove the KNOWN_MAX_TOKENS map. Hardcoding maxTokens values in
discovery is the wrong layer to solve this — any explicit value
still gets reserved against Bedrock's TPM quota at request start.
The correct fix is upstream in pi's Bedrock provider: omit maxTokens
from inferenceConfig when not explicitly set, letting the model use
its internal default. This avoids quota waste entirely.
See: badlogic/pi-mono#3399 and badlogic/pi-mono#3400
Keep the expanded KNOWN_CONTEXT_WINDOWS (context windows ARE the
right thing to set in discovery — they affect compaction thresholds
and session management, not API-level quota reservation).
* docs: clarify why hardcoded context windows are needed
Bedrock's ListFoundationModels and GetFoundationModel APIs return no
token limit information — there is no Bedrock API to discover context
windows or max output tokens programmatically. Note that this table
should become a fallback if AWS adds token metadata in the future.
* fix: add au and apac to inference profile prefix regex
Add missing geo prefixes discovered by querying inference profiles
across multiple regions:
- au. (Australia/NZ, used in ap-southeast-2/4/6)
- apac. (Asia-Pacific, used for older models in ap-northeast-1)
Both resolveKnownContextWindow and resolveBaseModelId now handle
all known prefixes: us, eu, ap, apac, au, jp, global.
* test: port au. prefix test from #65449 by @alickgithub2, add apac. coverage
Port the Australia/NZ inference profile test from PR #65449
(credit: @alickgithub2) and extend it to also cover the apac.
prefix discovered in ap-northeast-1.
* expand model coverage: Llama 4, MiniMax, NVIDIA, Mistral 3, GLM, Qwen
Cross-referenced KNOWN_CONTEXT_WINDOWS against live
list-foundation-models API. Added missing models:
- Llama 4 Maverick (1M) and Scout (512K)
- MiniMax M2/M2.1/M2.5 (1M)
- NVIDIA Nemotron Super/Nano variants (128K)
- Mistral Large 3 675B (128K)
- GLM 4.7/4.7-flash/5 (128K)
- Qwen3 Coder/32B/VL (128-256K)
Removed deprecated deepseek.v3-v1:0 and claude-opus-4-20250514
(not in active foundation models list).
* raise default context window from 128K to 200K
200K matches the floor for all current Claude models (the most
popular on Bedrock). Every other active model with a lower actual
limit is already in the explicit table. This ensures new Claude
models get a correct default without requiring a table update.
* test: update discovery test expectations for known context window values
* test: fix remaining contextWindow expectation (default 200K)
* fix(amazon-bedrock): keep conservative context fallback
* docs(changelog): note Bedrock context window fix
* fix(amazon-bedrock): normalize known context fallback
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
* feat(bedrock): add inference profile discovery and region injection
Inference profiles (cross-region and application) work with ConverseStream
but require the SDK client region to match the profile region. Without
this, users get "The provided model identifier is invalid" errors when
using cross-region profiles like us.anthropic.claude-sonnet-4-6.
Changes:
1. Inference profile discovery (discovery.ts):
- Call ListInferenceProfiles alongside ListFoundationModels (parallel)
- Inference profiles INHERIT capabilities from their underlying
foundation model (modalities, reasoning, context window, cost)
- resolveBaseModelId() maps profile → foundation model:
"us.anthropic.claude-sonnet-4-6" → "anthropic.claude-sonnet-4-6"
Application ARNs → extract model ID from models[].modelArn
- Graceful degradation if IAM lacks bedrock:ListInferenceProfiles
- Provider filter applies to profiles via underlying model ARNs
2. Region injection (register.sync.runtime.ts):
- Extract region from provider baseUrl or bedrockDiscovery.region
- Pass through to pi-ai options.region in wrapStreamFn
- Ensures SDK client connects to correct regional endpoint
3. Inference profile model detection (anthropic-family-cache-semantics.ts):
- isAnthropicBedrockModel() now recognizes application inference
profile ARNs (arn:aws:bedrock:...:application-inference-profile/*)
4. Tests (discovery.test.ts):
- New: inference profile inheritance test (4 models: 1 foundation +
3 profiles, verifies capability inheritance, inactive filtering)
- New: graceful AccessDeniedException handling test
- Updated: all existing tests for dual-API discovery pattern
Fixes#55642
* fix(bedrock): preserve inference profile model lookup
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
* fix(bedrock): stop injecting fake apiKey marker for aws-sdk auth when no env vars exist
When the Bedrock provider uses auth: "aws-sdk" and no AWS environment
variables are set (EC2 instance roles, ECS task roles, etc.),
resolveAwsSdkApiKeyVarName() fell back to "AWS_PROFILE" unconditionally.
This string was injected as apiKey in the provider config during
normalisation, which poisoned the downstream auth resolver — it treated
the marker as a literal key and failed with "No API key found".
The fix:
- resolveAwsSdkApiKeyVarName() now returns undefined (not "AWS_PROFILE")
when no AWS env vars are present
- resolveBedrockConfigApiKey() (extension) gets the same fix
- resolveMissingProviderApiKey() guards both the providerApiKeyResolver
and direct aws-sdk branches: if the resolver returns nothing, the
provider config is returned unchanged (no apiKey injected)
- The aws-sdk credential chain then resolves credentials at request time
via IMDS/ECS task role/etc. as intended
When AWS env vars ARE present (AWS_ACCESS_KEY_ID, AWS_PROFILE,
AWS_BEARER_TOKEN_BEDROCK), the marker is still injected correctly.
Closes#49891Closes#50699Fixes#54274
* test(bedrock): update resolveBedrockConfigApiKey test for undefined return on empty env
The test previously expected "AWS_PROFILE" when no env vars are set.
Now expects undefined (matching the fix), and adds a separate assertion
that AWS_PROFILE is returned when the env var is actually present.
* fix(bedrock): lock aws-sdk env marker behavior
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
