Skip to content

[Reasoning Engine] Agent Gateway returns 401 with runtime-bound Agent Identity token; succeeds when token binding is disabled #7209

Open

Description

@aman-swarnkar-cloud

Product / Component

Gemini Enterprise Agent Platform

Relevant areas:

  • Agent Runtime
  • Agent Identity
  • Agent Gateway — Agent-to-Anywhere
  • Context-Aware Access / certificate-bound tokens
  • Vertex AI Gemini inference

Description

I am seeing a reproducible authentication failure when using an Agent Runtime with:

  • Agent Identity
  • Agent-to-Anywhere Agent Gateway
  • default runtime-bound token / CAA protection enabled
  • Gemini inference through Vertex AI

The same deployed Runtime, Agent Identity, Gateway, IAM configuration, model, and application work successfully when runtime-bound token protection is explicitly disabled.

The failure appears to occur after Agent Gateway intercepts and authorizes the request, when the request is forwarded toward the Google API endpoint.

I would like confirmation whether this is:

  1. a known limitation of Agent Runtime + Agent-to-Anywhere Gateway with runtime-bound Agent Identity tokens,
  2. a configuration requirement I am missing, or
  3. an issue in the mTLS / bound-token / Agent Gateway → DPoP authentication handoff.

Architecture

ADK Agent
   ↓
Google Cloud Agent Runtime
   ↓
Unique Agent Identity
   ↓
Agent-to-Anywhere Agent Gateway
   ↓
IAP authorization
   ↓
Vertex AI Gemini

and

Agent Runtime
   ↓
Agent Identity
   ↓
Agent Gateway
   ↓
registered MCP server
   ↓
Cloud Run

The Agent Gateway is also being used to govern MCP access with IAM Unified Access Policies.

Environment

  • Agent Runtime region: us-central1
  • Gemini inference location: global
  • Model: gemini-3.8-flash
  • Gemini API version: v1
  • Agent Gateway mode: Agent-to-Anywhere
  • Runtime identity type: AGENT_IDENTITY
  • Google ADK: 2.10.0
  • Agent Platform SDK: 2.0.1
  • google-cloud-aiplatform: 2.0.1
  • google-genai: 2.29.0
  • MCP SDK: 2.3.0
  • Python runtime: Python 3.12

The Agent Identity has the documented Runtime/Gateway IAM prerequisites, including Agent Platform access, Agent Registry viewer access, and IAP egress access on registered destinations.

Reproduction

1. Deploy an Agent Runtime with Agent Identity

The runtime is deployed with:

"identity_type": types.IdentityType.AGENT_IDENTITY,

"agent_gateway_config": {
    "agent_to_anywhere_config": {
        "agent_gateway": GATEWAY_RESOURCE
    }
}

Do not opt out of runtime-bound-token protection.

2. Configure Gemini through Vertex AI

Gemini(
    model="gemini-3.8-flash",
    api_version="v1",
    client_kwargs={
        "enterprise": True,
        "location": "global",
    },
)

3. Invoke the deployed Runtime

The Runtime fails before producing a model response:

401 UNAUTHENTICATED

Request had invalid authentication credentials.
Expected OAuth 2 access token, login cookie or other
valid authentication credential.

4. Observe Agent Gateway request

Gateway logs show the intercepted destination:

aiplatform.mtls.googleapis.com

and:

authzPolicyInfo.result = ALLOWED
requestWasTlsIntercepted = true
clientCertPresent = true

The request therefore reaches Agent Gateway and passes its authorization evaluation, but Vertex AI returns 401 UNAUTHENTICATED.

5. Control test without Agent Gateway

An equivalent Agent Runtime configuration without the Gateway successfully invokes Gemini.

A direct Vertex AI call using the same project, model, global location, and v1 API also succeeds.

Therefore the model, project, location, and basic Vertex AI authorization are functional.

6. Disable runtime-bound-token protection

For diagnostic purposes, set the documented opt-out before Google authentication initializes:

GOOGLE_API_PREVENT_AGENT_TOKEN_SHARING_FOR_GCP_SERVICES=False

The Runtime diagnostics then show:

agent_identity_cert_present=False
bound_token_requested=False

7. Invoke the exact same Runtime again

The same Runtime now succeeds immediately:

model_version: gemini-3.8-flash
response: BOUND_TOKEN_DISABLED_OK

No other Runtime, Agent Identity, Gateway, model, or IAM configuration was changed.

8. Validate downstream MCP traffic

With bound-token protection disabled, the agent successfully performs an MCP call:

read_customer_profile(customer_id="CUST-1001")

and receives the expected response.

Agent Gateway logs confirm that the call traversed the Gateway:

mcpInfo.method = tools/call
mcpInfo.parameter = read_customer_profile

authzPolicyInfo.result = ALLOWED
requestWasTlsIntercepted = true

The IAP audit log also identifies the Runtime's Agent Identity and shows:

permission = iap.resources.egressViaIAP
granted = true

This confirms that the working request still uses:

Agent Runtime
→ Agent Identity
→ Agent Gateway
→ downstream resource

A/B result

The behavior is consistently:

Same Runtime
Same Agent Identity
Same Agent Gateway
Same IAM permissions
Same Agent Registry configuration
Same model
Same Vertex AI endpoint
Same application

Runtime-bound token protection ENABLED
→ Agent Gateway receives request
→ Gateway authorization ALLOWED
→ aiplatform.mtls.googleapis.com
→ 401 UNAUTHENTICATED

Runtime-bound token protection DISABLED
→ same architecture
→ Gemini succeeds
→ MCP calls through Agent Gateway succeed

Additional diagnostics

The Python GenAI transport reports:

aiohttp=True
google_auth_async=True
client_certificate_requested=True
certificate_source_available=True

Therefore the SDK appears to be selecting its mTLS-aware asynchronous Google authentication path.

The Runtime is also registered correctly with Agent Registry and has IAP egress access to its registered destinations.

Expected behavior

Based on the documented Context-Aware Access Agent Security architecture, I expected the following flow to work without opting out of runtime-bound-token protection:

Agent Runtime / Agent Identity
→ certificate-bound token + mTLS
→ Agent Gateway
→ DPoP reauthentication
→ Google Cloud API

In particular, the documentation states that because mTLS terminates at Agent Gateway, Agent Gateway uses DPoP when reauthenticating the agent toward Google Cloud APIs.

Actual behavior

With runtime-bound-token protection enabled:

Agent Gateway authorization: ALLOWED
TLS interception: true
Destination: aiplatform.mtls.googleapis.com
Result: 401 UNAUTHENTICATED

With the documented CAA/bound-token opt-out:

Result: success

Documentation concern

The current Agent Runtime → Agent Gateway deployment example explicitly configures:

GOOGLE_API_PREVENT_AGENT_TOKEN_SHARING_FOR_GCP_SERVICES=False

for Agent-to-Anywhere deployments.

At the same time, the Context-Aware Access Agent Security documentation describes Agent Gateway using DPoP after mTLS termination to preserve proof-of-possession when accessing Google APIs.

Could you clarify whether disabling runtime-bound tokens is currently an expected requirement for Agent-to-Anywhere Agent Gateway, or whether the bound-token → Gateway → DPoP path should work without this opt-out?

Security impact

The documented workaround restores functionality but removes an important security property.

With bound-token protection enabled, possession of the access token alone should not be sufficient to replay the credential outside its intended Runtime context.

With the protection disabled, IAM, Agent Gateway, and UAP authorization still restrict what the Agent Identity can access, but a stolen bearer token could potentially be replayed during its validity period.

For a production architecture, I would prefer to retain both:

Agent Identity + certificate-bound token / DPoP

and

Agent Gateway + IAM/UAP least-privilege authorization

rather than treating UAP as a replacement for proof-of-possession.

Related public issues

There appear to be related, although not identical, reports:

  • google/adk-python#5361 — MCP Toolset incompatibility with mTLS / bound tokens resulting in 401
  • google/adk-python#6365 — Agent Runtime using Agent Identity receives 401 for outbound A2A requests while service-account execution succeeds

I could not find a public report reproducing this exact combination:

Agent Runtime
+ Agent Identity
+ Agent-to-Anywhere Agent Gateway
+ Vertex AI Gemini
+ runtime-bound token enabled

Questions for Google

  1. Should Agent Runtime + Agent Identity + Agent-to-Anywhere Agent Gateway support Vertex AI access while runtime-bound-token protection remains enabled?

  2. If yes, is there an additional configuration required for the Gateway to perform the documented DPoP reauthentication toward aiplatform.mtls.googleapis.com?

  3. If no, is disabling runtime-bound-token protection currently a known limitation/required workaround for Agent-to-Anywhere deployments?

  4. Is there an ETA or tracked issue for supporting this path without opting out of CAA/token binding?

  5. For production workloads using Agent Gateway and UAP, what security posture does Google currently recommend while this limitation exists?

Reproducibility

Occurrence rate: 100% in this environment

Bound token enabled  → reproducible 401
Bound token disabled → reproducible success

I can provide sanitized Agent Gateway request logs, Runtime authentication diagnostics, Agent Registry configuration, and the minimal public reproduction repository if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    api: vertex-aiIssues related to the googleapis/python-aiplatform API.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions