Repository navigation
[Reasoning Engine] Agent Gateway returns 401 with runtime-bound Agent Identity token; succeeds when token binding is disabled #7209
Description
Product / Component
Gemini Enterprise Agent Platform
Relevant areas:
- Agent Runtime
- Agent Identity
- Agent Gateway — Agent-to-Anywhere
- Context-Aware Access / certificate-bound tokens
- Vertex AI Gemini inference
Description
I am seeing a reproducible authentication failure when using an Agent Runtime with:
- Agent Identity
- Agent-to-Anywhere Agent Gateway
- default runtime-bound token / CAA protection enabled
- Gemini inference through Vertex AI
The same deployed Runtime, Agent Identity, Gateway, IAM configuration, model, and application work successfully when runtime-bound token protection is explicitly disabled.
The failure appears to occur after Agent Gateway intercepts and authorizes the request, when the request is forwarded toward the Google API endpoint.
I would like confirmation whether this is:
- a known limitation of Agent Runtime + Agent-to-Anywhere Gateway with runtime-bound Agent Identity tokens,
- a configuration requirement I am missing, or
- an issue in the mTLS / bound-token / Agent Gateway → DPoP authentication handoff.
Architecture
ADK Agent
↓
Google Cloud Agent Runtime
↓
Unique Agent Identity
↓
Agent-to-Anywhere Agent Gateway
↓
IAP authorization
↓
Vertex AI Gemini
and
Agent Runtime
↓
Agent Identity
↓
Agent Gateway
↓
registered MCP server
↓
Cloud Run
The Agent Gateway is also being used to govern MCP access with IAM Unified Access Policies.
Environment
- Agent Runtime region:
us-central1 - Gemini inference location:
global - Model:
gemini-3.8-flash - Gemini API version:
v1 - Agent Gateway mode: Agent-to-Anywhere
- Runtime identity type:
AGENT_IDENTITY - Google ADK:
2.10.0 - Agent Platform SDK:
2.0.1 - google-cloud-aiplatform:
2.0.1 - google-genai:
2.29.0 - MCP SDK:
2.3.0 - Python runtime: Python 3.12
The Agent Identity has the documented Runtime/Gateway IAM prerequisites, including Agent Platform access, Agent Registry viewer access, and IAP egress access on registered destinations.
Reproduction
1. Deploy an Agent Runtime with Agent Identity
The runtime is deployed with:
"identity_type": types.IdentityType.AGENT_IDENTITY,
"agent_gateway_config": {
"agent_to_anywhere_config": {
"agent_gateway": GATEWAY_RESOURCE
}
}Do not opt out of runtime-bound-token protection.
2. Configure Gemini through Vertex AI
Gemini(
model="gemini-3.8-flash",
api_version="v1",
client_kwargs={
"enterprise": True,
"location": "global",
},
)3. Invoke the deployed Runtime
The Runtime fails before producing a model response:
401 UNAUTHENTICATED
Request had invalid authentication credentials.
Expected OAuth 2 access token, login cookie or other
valid authentication credential.
4. Observe Agent Gateway request
Gateway logs show the intercepted destination:
aiplatform.mtls.googleapis.com
and:
authzPolicyInfo.result = ALLOWED
requestWasTlsIntercepted = true
clientCertPresent = true
The request therefore reaches Agent Gateway and passes its authorization evaluation, but Vertex AI returns 401 UNAUTHENTICATED.
5. Control test without Agent Gateway
An equivalent Agent Runtime configuration without the Gateway successfully invokes Gemini.
A direct Vertex AI call using the same project, model, global location, and v1 API also succeeds.
Therefore the model, project, location, and basic Vertex AI authorization are functional.
6. Disable runtime-bound-token protection
For diagnostic purposes, set the documented opt-out before Google authentication initializes:
GOOGLE_API_PREVENT_AGENT_TOKEN_SHARING_FOR_GCP_SERVICES=False
The Runtime diagnostics then show:
agent_identity_cert_present=False
bound_token_requested=False
7. Invoke the exact same Runtime again
The same Runtime now succeeds immediately:
model_version: gemini-3.8-flash
response: BOUND_TOKEN_DISABLED_OK
No other Runtime, Agent Identity, Gateway, model, or IAM configuration was changed.
8. Validate downstream MCP traffic
With bound-token protection disabled, the agent successfully performs an MCP call:
read_customer_profile(customer_id="CUST-1001")
and receives the expected response.
Agent Gateway logs confirm that the call traversed the Gateway:
mcpInfo.method = tools/call
mcpInfo.parameter = read_customer_profile
authzPolicyInfo.result = ALLOWED
requestWasTlsIntercepted = true
The IAP audit log also identifies the Runtime's Agent Identity and shows:
permission = iap.resources.egressViaIAP
granted = true
This confirms that the working request still uses:
Agent Runtime
→ Agent Identity
→ Agent Gateway
→ downstream resource
A/B result
The behavior is consistently:
Same Runtime
Same Agent Identity
Same Agent Gateway
Same IAM permissions
Same Agent Registry configuration
Same model
Same Vertex AI endpoint
Same application
Runtime-bound token protection ENABLED
→ Agent Gateway receives request
→ Gateway authorization ALLOWED
→ aiplatform.mtls.googleapis.com
→ 401 UNAUTHENTICATED
Runtime-bound token protection DISABLED
→ same architecture
→ Gemini succeeds
→ MCP calls through Agent Gateway succeed
Additional diagnostics
The Python GenAI transport reports:
aiohttp=True
google_auth_async=True
client_certificate_requested=True
certificate_source_available=True
Therefore the SDK appears to be selecting its mTLS-aware asynchronous Google authentication path.
The Runtime is also registered correctly with Agent Registry and has IAP egress access to its registered destinations.
Expected behavior
Based on the documented Context-Aware Access Agent Security architecture, I expected the following flow to work without opting out of runtime-bound-token protection:
Agent Runtime / Agent Identity
→ certificate-bound token + mTLS
→ Agent Gateway
→ DPoP reauthentication
→ Google Cloud API
In particular, the documentation states that because mTLS terminates at Agent Gateway, Agent Gateway uses DPoP when reauthenticating the agent toward Google Cloud APIs.
Actual behavior
With runtime-bound-token protection enabled:
Agent Gateway authorization: ALLOWED
TLS interception: true
Destination: aiplatform.mtls.googleapis.com
Result: 401 UNAUTHENTICATED
With the documented CAA/bound-token opt-out:
Result: success
Documentation concern
The current Agent Runtime → Agent Gateway deployment example explicitly configures:
GOOGLE_API_PREVENT_AGENT_TOKEN_SHARING_FOR_GCP_SERVICES=False
for Agent-to-Anywhere deployments.
At the same time, the Context-Aware Access Agent Security documentation describes Agent Gateway using DPoP after mTLS termination to preserve proof-of-possession when accessing Google APIs.
Could you clarify whether disabling runtime-bound tokens is currently an expected requirement for Agent-to-Anywhere Agent Gateway, or whether the bound-token → Gateway → DPoP path should work without this opt-out?
Security impact
The documented workaround restores functionality but removes an important security property.
With bound-token protection enabled, possession of the access token alone should not be sufficient to replay the credential outside its intended Runtime context.
With the protection disabled, IAM, Agent Gateway, and UAP authorization still restrict what the Agent Identity can access, but a stolen bearer token could potentially be replayed during its validity period.
For a production architecture, I would prefer to retain both:
Agent Identity + certificate-bound token / DPoP
and
Agent Gateway + IAM/UAP least-privilege authorization
rather than treating UAP as a replacement for proof-of-possession.
Related public issues
There appear to be related, although not identical, reports:
google/adk-python#5361— MCP Toolset incompatibility with mTLS / bound tokens resulting in 401google/adk-python#6365— Agent Runtime using Agent Identity receives 401 for outbound A2A requests while service-account execution succeeds
I could not find a public report reproducing this exact combination:
Agent Runtime
+ Agent Identity
+ Agent-to-Anywhere Agent Gateway
+ Vertex AI Gemini
+ runtime-bound token enabled
Questions for Google
-
Should Agent Runtime + Agent Identity + Agent-to-Anywhere Agent Gateway support Vertex AI access while runtime-bound-token protection remains enabled?
-
If yes, is there an additional configuration required for the Gateway to perform the documented DPoP reauthentication toward
aiplatform.mtls.googleapis.com? -
If no, is disabling runtime-bound-token protection currently a known limitation/required workaround for Agent-to-Anywhere deployments?
-
Is there an ETA or tracked issue for supporting this path without opting out of CAA/token binding?
-
For production workloads using Agent Gateway and UAP, what security posture does Google currently recommend while this limitation exists?
Reproducibility
Occurrence rate: 100% in this environment
Bound token enabled → reproducible 401
Bound token disabled → reproducible success
I can provide sanitized Agent Gateway request logs, Runtime authentication diagnostics, Agent Registry configuration, and the minimal public reproduction repository if useful.
Activity