support mantle on existing bedrock providers

[nfuden]

Seconds attempt to add mantle support and ways to either target only mantle, runtime or a fallthrough.

Note that the ability to set foundational models for runtime is annoying to set generally as an org or subsection of an org can massage the allow lists, therefore having the ability to override the default lists of models seems valuable. In order to enable this added an agctl command to quickly refresh our lists but also we can spend more time to get this better while still allowing for specific bedrock provider setups to have just mantle or just bedrock.

Note that scope of this seemed to blow up more than expected especially with regards to the different endpoints that mantle supports like the anthropic path.

This also infects some of our main route / auth work due to decisions not being purely on provider but also checking the model itself.

I've vetted this with
AGENTGATEWAY_E2E=true cargo test --test integration -- --ignored mantle_smoke

and checked that the bedrock messages

AGENTGATEWAY_E2E=true cargo test --test integration tests::llm::bedrock::messages

Some thoughts:
Q:Do we really need to have an external download with scraping stuff?
A: probably not but this allows for easy update to bedrock model lists without doing the aws cli command as that requires authentication to an aws account... which we could optimize for but then wouldnt test in ci.

Q: Shouldnt this just be controlled via a catalog?
A: We only really have a costcatalog and it doesnt quite hit our form factor and is nicely tied to models.dev which does not contain the info we need for runtime supported models.

Side note that for these ive had to manually set rustminstack but that was true even before changes.

[Copilot]

Pull request overview

Adds AWS Bedrock “Mantle” support alongside existing Bedrock Runtime routing by introducing a configurable provider preference (runtime vs mantle), plus a Mantle-only model allow-list that can be loaded/overridden at runtime and refreshed via a CLI import command. This extends the gateway’s Bedrock integration to support additional Mantle-native endpoints/paths (including Anthropic-style paths) while keeping non-chat routes on Runtime.

Changes:

• Introduces BedrockProviderPreference (runtimePreferred/mantleOnly/runtimeOnly) and routes Bedrock requests to Runtime vs Mantle hosts/paths (including SigV4 service override for Mantle).
• Adds a global Mantle allow-list model table with hot-reload support and config/env wiring (bedrockMantleModelCatalog).
• Adds agctl import bedrock CLI functionality to scrape AWS docs and generate the Mantle-only model allow-list JSON.

Reviewed changes

Copilot reviewed 33 out of 35 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
schema/config.md	Documents new Bedrock Mantle model catalog sources and provider preference fields (plus other schema doc updates).
schema/config.json	Adds bedrockMantleModelCatalog and Bedrock provider preference schema definitions/defaults.
crates/protos/proto/resource.proto	Extends the Bedrock provider proto with provider_preference and its enum.
crates/llm/src/lib.rs	Exposes the new Bedrock model table module.
crates/llm/src/golden_tests.rs	Updates golden tests to populate the new provider_preference field.
crates/llm/src/conversion/rerank_tests.rs	Updates rerank conversion tests for Bedrock host resolution signature changes.
crates/llm/src/conversion/bedrock_tests.rs	Updates Bedrock conversion tests to include provider_preference.
crates/llm/src/bedrock.rs	Implements endpoint resolution (Runtime vs Mantle), Mantle host/path selection, and signing service override.
crates/llm/src/bedrock_model_table.rs	Adds a global, swappable Mantle allow-list table and JSON parsing helpers.
crates/llm/src/bedrock_mantle_models.json	Adds embedded default allow-list payload (currently empty).
crates/llm/Cargo.toml	Adds arc-swap dependency for global allow-list swapping.
crates/agentgateway/tests/tests/llm_providers.rs	Adds E2E smoke test coverage for Mantle-only routing and config helper changes.
crates/agentgateway/src/types/local.rs	Adds local LLM param wiring for Bedrock provider preference into provider config.
crates/agentgateway/src/types/agent_xds.rs	Maps proto provider_preference into runtime Bedrock provider preference.
crates/agentgateway/src/proxy/httpproxy.rs	Ensures Bedrock connection target aligns with model-aware authority when Mantle/Runtime differ.
crates/agentgateway/src/llm/tests.rs	Adds Bedrock Mantle routing/signing tests and updates existing Bedrock tests for new fields.
crates/agentgateway/src/llm/mod.rs	Makes Bedrock supported chat formats model-aware and adds Mantle catalog initialization module.
crates/agentgateway/src/llm/bedrock_mantle_catalog.rs	Implements startup load + file watch hot-reload for the Mantle allow-list.
crates/agentgateway/src/lib.rs	Adds config structs for bedrock_mantle_model_catalog to the main config.
crates/agentgateway/src/config.rs	Adds env/config parsing for BEDROCK_MANTLE_MODEL_CATALOG_PATHS.
crates/agentgateway/src/app.rs	Initializes Bedrock Mantle allow-list loading on startup.
controller/pkg/syncer/backend/testdata/Valid_Bedrock_backend_with_RuntimeOnly_provider_preference.yaml	Adds testdata for RuntimeOnly preference translation.
controller/pkg/syncer/backend/testdata/Valid_Bedrock_backend_with_MantleOnly_provider_preference.yaml	Adds testdata for MantleOnly preference translation.
controller/pkg/syncer/backend/testdata/Valid_Bedrock_backend_with_default_provider_preference.yaml	Adds testdata for default preference translation.
controller/pkg/syncer/backend/backend_plugin.go	Translates Bedrock provider preference from CRD into proto enum.
controller/pkg/syncer/backend/backend_plugin_test.go	Adds tests for Bedrock provider preference translation cases.
controller/pkg/cli/root.go	Registers the new import command group in agctl.
controller/pkg/cli/imports/cmd.go	Adds agctl import command group scaffolding.
controller/pkg/cli/imports/bedrock/source_models.go	Implements AWS docs scraping/parsing to derive Mantle-only model IDs.
controller/pkg/cli/imports/bedrock/source_models_test.go	Adds parser unit tests and a live scrape test.
controller/pkg/cli/imports/bedrock/cmd.go	Adds agctl import bedrock command implementation and JSON output.
controller/install/helm/agentgateway-crds/templates/agentgateway.dev_agentgatewaybackends.yaml	Extends CRD schema to support providerPreference on Bedrock config.
controller/api/v1alpha1/agentgateway/agentgateway_backend_types.go	Adds Bedrock provider preference enum/type to the CRD Go types.
Cargo.lock	Locks arc-swap dependency addition.
api/resource.pb.go	Regenerates Go protobuf output for the new Bedrock provider preference enum/field.

Comments suppressed due to low confidence (1)

controller/pkg/cli/imports/bedrock/source_models.go:137

• Same bufio.Scanner token-limit risk as above: model cards can contain long lines; without increasing the buffer, the scan can fail and silently drop IDs.

	scanner := bufio.NewScanner(r)
	for scanner.Scan() {

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.