Agent Gateway

The VeraID Agent Gateway is a transparent control plane that sits between your AI agents and their LLM providers. Every prompt, tool call, and model response flows through the gateway, where VeraID enforces budget limits, scans for prompt injection, evaluates access policies, and logs the interaction for audit — all before the request reaches the LLM or the response reaches your agent.

Why a Gateway

Governing AI agents at the application layer alone is insufficient. Agents can be modified, forked, or misconfigured in ways that bypass application-level checks. The gateway operates at the network layer, ensuring that no LLM call from a governed agent can bypass policy evaluation, regardless of how the agent is implemented.

Key benefits:

• Centralized enforcement — Security controls apply uniformly across all agents, frameworks, and providers
• Zero trust for AI — Every request is authenticated, authorized, and audited
• Framework-agnostic — Works with any agent that makes HTTP calls to an LLM API
• Real-time cost control — Budget checks happen before tokens are consumed, not after

---

Gateway Request Flow

When an AI agent makes an LLM API call through the VeraID SDK, the request passes through a multi-stage pipeline before reaching the provider.

┌─────────────┐     ┌───────────────────────────────────────────────────┐     ┌─────────────┐
│             │     │                 VeraID Gateway                    │     │             │
│   AI Agent  │────►│                                                   │────►│  LLM API    │
│             │     │  ┌──────────┐  ┌──────────┐  ┌────────────────┐  │     │  (OpenAI,   │
│             │◄────│  │  Prompt  │  │  Budget  │  │    Policy      │  │◄────│  Anthropic, │
│             │     │  │ Analysis │─►│  Check   │─►│  Evaluation    │  │     │  Azure)     │
│             │     │  └──────────┘  └──────────┘  └────────────────┘  │     │             │
│             │     │       │              │              │             │     │             │
│             │     │       ▼              ▼              ▼             │     │             │
│             │     │  ┌───────────────────────────────────────────┐    │     │             │
│             │     │  │           Audit Log + Cost Tracker        │    │     │             │
│             │     │  └───────────────────────────────────────────┘    │     │             │
│             │     │                                                   │     │             │
└─────────────┘     └───────────────────────────────────────────────────┘     └─────────────┘

Stage 1: Prompt Analysis

The gateway inspects the inbound prompt for adversarial content before it reaches the LLM. This includes scanning for instruction overrides, role manipulation attempts, data exfiltration patterns, and known jailbreak techniques.

• If a high-risk injection is detected, the request is blocked and the agent receives an error response
• Medium-risk detections are flagged and optionally routed for human review
• Low-risk or clean prompts proceed to the next stage
• All detections are logged with risk scores and pattern details

See Prompt Injection Detection for the full detection pipeline.

Stage 2: Budget Check

The gateway verifies that the agent has sufficient budget to complete the request. Budget checks evaluate three limits:

1. Per-request limit — Estimated cost of this individual request based on token count and model pricing
2. Daily limit — Cumulative spend for the current UTC day
3. Monthly limit — Cumulative spend for the current calendar month

If any limit would be exceeded, the gateway applies the configured action: block the request, throttle by queuing it for later, or alert administrators while allowing the request to proceed.

See Budget Controls for configuration details.

Stage 3: Policy Evaluation

The gateway evaluates all PBAC policies that apply to the requesting agent identity. This includes:

• Resource access — Is the agent permitted to call this specific model endpoint?
• Action authorization — Is the agent allowed to perform this operation type (chat completion, function calling, image generation)?
• Condition evaluation — Are runtime conditions met (time windows, IP restrictions, rate limits)?
• Data access controls — Does the prompt reference data sources outside the agent’s dataAccess.allowed list?

If policy evaluation returns a DENY result, the request is blocked with a structured error explaining which policy denied access.

Stage 4: Forward to LLM

Requests that pass all three checks are forwarded to the configured LLM provider. The gateway:

• Injects the agent’s managed credentials (API keys, tokens) from the VeraID vault
• Strips any VeraID-specific headers before forwarding
• Maintains the original request format expected by the provider’s API
• Handles retries and failover according to the agent’s configuration

Stage 5: Log Response

When the LLM returns a response, the gateway captures it for audit and cost tracking before forwarding it to the agent.

• Token usage — Input tokens, output tokens, and total tokens are recorded
• Cost calculation — Actual cost is computed based on the provider’s pricing for the specific model
• Response logging — If monitoring.logResponses is enabled, the full response body is stored in the audit log
• Cumulative tracking — Daily and monthly spend counters are updated in real time

Stage 6: Track Cost

The final stage updates the agent’s budget state and evaluates warning thresholds.

• If spend exceeds the warning threshold (e.g., 80% of daily limit), an alert is dispatched to configured notification channels
• Budget state is persisted in Redis for low-latency reads on subsequent requests
• Cost data is written to TimescaleDB for historical analytics and reporting

---

Architecture

The gateway is deployed as part of the VeraID API layer on Cloud Run. It does not require a separate service or infrastructure component.

SDK Connection

Agents connect to the gateway through the VeraID SDK, which transparently routes LLM API calls through the gateway.

import { VeraIDClient } from '@veraid/sdk';

const client = new VeraIDClient({
  apiKey: process.env.VERAID_API_KEY,
  agentId: process.env.VERAID_AGENT_ID,
});

// The SDK routes this call through the VeraID gateway
const response = await client.agents.complete({
  model: 'gpt-4o',
  messages: [
    { role: 'system', content: 'You are a helpful assistant.' },
    { role: 'user', content: userInput },
  ],
  temperature: 0.7,
});

Direct API Integration

For agents that cannot use the SDK, the gateway exposes a REST endpoint that mirrors the LLM provider’s API format.

# Route an OpenAI-compatible request through the VeraID gateway
curl -X POST https://app.veraid.io/api/v1/gateway/openai/chat/completions \
  -H "Authorization: Bearer $VERAID_API_KEY" \
  -H "X-Agent-Id: $VERAID_AGENT_ID" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "Summarize the Q3 report."}
    ]
  }'

The gateway translates between VeraID authentication and the downstream provider’s authentication, so your agents never hold raw LLM API keys.

Tip

Using the gateway endpoint means your agents never need direct access to OpenAI, Anthropic, or Azure API keys. VeraID manages, rotates, and injects provider credentials automatically — reducing your secret sprawl and eliminating the risk of leaked LLM API keys.

Supported Providers

The gateway supports the following LLM providers:

Provider	Gateway Endpoint	API Compatibility
OpenAI	/api/v1/gateway/openai/*	Full OpenAI API compatibility
Anthropic	/api/v1/gateway/anthropic/*	Full Anthropic Messages API compatibility
Azure OpenAI	/api/v1/gateway/azure/*	Full Azure OpenAI API compatibility
Custom	/api/v1/gateway/custom/*	Configurable proxy for self-hosted or alternative providers

---

Gateway vs. Direct Integration

Capability	Gateway	Direct SDK Only
Prompt injection scanning	Pre-request, blocks before LLM call	Post-analysis, after tokens consumed
Budget enforcement	Pre-request, prevents overspend	Best-effort, race conditions possible
Credential management	Automatic injection, zero agent access to raw keys	Agent holds credentials directly
Audit logging	Guaranteed, gateway-level capture	Dependent on SDK integration correctness
Framework compatibility	Any HTTP-based agent	SDK-supported frameworks only

Caution

Direct SDK integration without the gateway provides weaker guarantees. Budget checks and prompt scanning happen in the agent’s process, which means a compromised or misconfigured agent could bypass them. The gateway enforces controls at the network level, outside the agent’s trust boundary.

---

What’s Next

• Budget Controls — Configure spend limits and cost tracking
• Prompt Injection Detection — Understand the detection pipeline in detail
• Approval Workflows — Add human-in-the-loop review for sensitive operations