Switchyard Is What? Rust AI Gateway Guide (2026)

Switchyard AI Gateway is worth testing if you need one routing layer between Claude Code, Codex, and several model backends. Start with the Python proxy and CLI for agent integration; evaluate the separate Rust server only when its service boundary and configuration model match your deployment needs.

This week’s action: run one single-model passthrough first, then test protocol translation, fallback behavior, tool calls, and request statistics before adding intelligent routing.

This guide is for developers connecting Claude Code or Codex to different providers, platform teams building a shared AI Gateway, and engineers comparing a Python proxy with an independent Rust server.

Last updated August 14, 2026. Technical details were checked against the official Switchyard repository, installation material, architecture documentation, and Rust package structure.

Python proxy versus Rust server

The title needs one important qualification. Switchyard is not accurately described as a project that is entirely written in Rust.

The official repository describes Switchyard as a Python proxy for LLM traffic. Its main proxy and CLI workflow includes Python packages, Python installation commands, and launcher commands for Claude Code and Codex. At the same time, the repository contains a separate switchyard_rust area, Rust crates, a Cargo.toml, and documentation for a Rust switchyard-server binary.

That distinction changes how you evaluate it:

The practical decision: use the Python route to validate your agent workflow. Do not switch to the Rust server simply because the repository contains Cargo files.

The official NeMo documentation also presents Rust contracts and a separate server path. That supports a component-level description, not a claim that every Switchyard function is implemented in Rust.

Client protocol versus backend protocol

The core problem is not just “calling a model.” Your client and backend may speak different API dialects.

Claude Code commonly expects an Anthropic-style interaction. Other clients, SDKs, and local inference servers may expose OpenAI Chat Completions or OpenAI Responses. A backend can also add its own limits around tools, structured output, reasoning fields, streaming, or model naming.

Switchyard sits between these systems:

Claude Code / Codex / SDK
          |
          | OpenAI or Anthropic client format
          v
      Switchyard
          |
          | Backend-compatible request
          v
Hosted provider / vLLM / Ollama / Azure / private endpoint

The official architecture describes this as a stable client-facing interface combined with routing, translation, and configured fallback behavior. (Switchyard architecture documentation)

The documented protocol set includes:

The official repository lists OpenAI Chat Completions, Anthropic Messages, OpenAI Responses, and OpenAI-compatible APIs as supported areas. It also names common compatible backends such as vLLM, Ollama, and Azure. “Compatible,” however, does not mean that every provider-specific extension will pass through unchanged. (Switchyard protocol documentation)

Supported, experimental, and backend-dependent behavior

Treat protocol support in three layers.

Documented support: the three named API families and compatible OpenAI-style endpoints.

Configuration-dependent support: routing profiles, fallbacks, statistics, session affinity, and provider credentials. These work only when the selected profile and backend expose the required fields.

Backend-dependent behavior: tool calls, MCP bridges, long context, structured output, provider-specific headers, and special reasoning fields. These require a real request test.

A useful example is the Bedrock-backed Claude Code and MCP caveat documented by Switchyard. The project warns that Bedrock enforces a toolSpec.name length limit, while Claude Code’s MCP bridge may inject longer names. That means a request can succeed for plain text but fail once tools are included. (Switchyard compatibility notes)

Compatibility reminder: test at least one tool-bearing request. A successful /v1/models response proves that the endpoint is reachable, not that Claude Code, MCP, streaming, or structured output will work.

Single backend versus routed traffic

Switchyard supports more than simple forwarding, but routing should be added only after passthrough works.

The documented routing options include single-model passthrough, random routing, LLM-as-classifier routing, signal-driven stage routing, and custom routers. The launcher can also use a routing profile instead of a fixed model. (Switchyard routing documentation)

Do not assume that an intelligent router will automatically lower cost or improve answer quality. A classifier may add another model call. A weak model may be selected for a task it cannot complete. A fallback may repeat work on a different backend. The correct question is whether your measured quality, latency, and failure rate improve for your workload.

Session affinity matters when several turns belong to one agent task. If a classifier or stage router sees only isolated requests, it may make inconsistent decisions across a multi-turn session. The official documentation includes sticky routing and session-affinity material for this reason.

A safer rollout sequence is:

1. Pin one model.
2. Record latency, token usage, errors, and tool-call success.
3. Add a second backend with the same client-facing format.
4. Test random routing.
5. Add classifier or stage routing only with a fixed evaluation set.
6. Define fallback conditions explicitly.
7. Compare results against the pinned baseline.

For a broader view of how model-routing products differ, see this AI Gateway comparison guide before selecting a long-term architecture.

Launcher workflow for Claude Code and Codex

The Agent Launcher is designed to remove repetitive environment editing during local tests. The official quick start shows launcher commands for Claude Code and Codex. Each launcher starts a local proxy, points the target client at it, and shuts the proxy down when the client exits. (Switchyard launcher documentation)

The basic workflow is:

1. Prepare the Python environment

The documented installation path uses Python 3.12 or newer. Optional extras add server and CLI functionality. The repository lists separate installation options for the base package, server, CLI, and combined features.

pip install "nemo-switchyard[cli,server]"

If you prefer source installation, use the repository’s documented uv workflow rather than mixing package-manager environments without a reason.

2. Define the backend credential

Keep credentials outside shell history and source control.

export MODEL_API_KEY="replace-with-your-key"
export MODEL_BASE_URL="https://your-openai-compatible-endpoint/v1"

The exact environment variables depend on the provider and route configuration. Do not assume that a client-facing API key and an upstream provider key are interchangeable.

3. Start with one model

Use a fixed model first. This isolates protocol problems from routing problems.

switchyard launch claude \
  --model your-provider/model-name \
  --api-key "$MODEL_API_KEY" \
  --base-url "$MODEL_BASE_URL"

For Codex, use the corresponding switchyard launch codex command documented by the project. The important test is not whether the process opens. It is whether a real prompt completes with the expected response format.

4. Test tools and MCP separately

Run one plain-text prompt. Then run a tool call. Then run an MCP-backed task if your workflow depends on MCP.

Check:

• Tool name preservation.
• Input schema handling.
• Streaming behavior.
• Error propagation.
• Model alias selection.
• Context length under a realistic prompt.
• Whether the backend accepts the same tool format as the client.

5. Move to a routing profile

Once passthrough works, define the route in YAML. The official examples expose the route name as a model ID and allow the client to select it through the request’s model field.

defaults:
  api_key: ${MODEL_API_KEY}
  base_url: https://your-openai-compatible-endpoint/v1
  format: openai

routes:
  coding-route:
    type: random_routing
    strong:
      model: provider/strong-model
    weak:
      model: provider/weak-model
    strong_probability: 0.3

Use environment substitution for credentials. Review the profile before placing it on a shared host.

6. Test fallback and session behavior

Stop one backend deliberately. Send a request. Confirm whether the configured fallback activates, whether the error is visible to the client, and whether a multi-turn session remains coherent.

Do not call a fallback “reliable” until you have tested:

• Timeout.
• HTTP error.
• Invalid model response.
• Tool-call failure.
• Streaming interruption.
• Credential rejection.
• Backend recovery after restart.

Standalone proxy versus local launcher

Switchyard can be used as more than a one-off local wrapper. The official README documents a standalone Python server mode with a routing profile and an HTTP port. In that mode, the route name becomes a model ID that clients can request through the proxy. (Switchyard server documentation)

A standalone command may look like this:

switchyard --routing-profiles routes.yaml -- serve --port 4000

The project’s example then checks the model endpoint and submits an OpenAI-compatible request. That confirms the HTTP path, but it does not replace end-to-end agent testing.

For production-like use, separate these concerns:

• Credentials: upstream keys should not be exposed to every client.
• Configuration: route changes need review and rollback.
• Statistics: token, cost, latency, and error data must be retained in a useful form.
• Session affinity: multi-turn tasks may need consistent routing.
• Availability: the host needs restart supervision and a health endpoint.
• Permissions: only approved clients should reach the proxy.
• Capacity: the gateway can become a bottleneck even when model backends are healthy.

The Rust server deserves its own acceptance path. Its configuration schema is separate. Verify its supported endpoints, routing behavior, statistics, authentication model, and operational controls rather than assuming that the Python CLI and Rust binary expose identical flags or feature coverage.

Production acceptance checklist

Use this checklist before calling Switchyard a shared AI Gateway:

• [ ] One pinned model works through Claude Code.
• [ ] One pinned model works through Codex.
• [ ] OpenAI Chat Completions translation passes a real request.
• [ ] Anthropic Messages translation passes a real request.
• [ ] OpenAI Responses behavior is tested if your client requires it.
• [ ] Streaming output matches the client’s expectations.
• [ ] A tool-bearing request completes successfully.
• [ ] MCP behavior is tested separately from plain text.
• [ ] Provider-specific model names are mapped clearly.
• [ ] API keys are injected at runtime, not committed to YAML.
• [ ] A backend timeout produces a visible and actionable error.
• [ ] Fallback behavior is tested with the primary backend offline.
• [ ] Multi-turn sessions are tested for routing consistency.
• [ ] Request statistics include latency, token usage, and error status.
• [ ] Logs avoid leaking prompts, secrets, or sensitive tool arguments.
• [ ] The service has a restart policy and health check.
• [ ] Route changes can be reviewed and rolled back.
• [ ] Python and Rust components are documented as separate deployment choices.
• [ ] The exact Switchyard commit and package version are recorded.
• [ ] Your workload has a baseline without intelligent routing.

The final item is easy to skip. Without a baseline, you cannot tell whether routing improved the system or merely added another failure surface.

If your main goal is improving an agent coding workflow rather than operating a gateway, this guide to Claude Code skills and workflow structure can help you separate agent behavior problems from model-routing problems.

FAQ: Switchyard deployment decisions

What is Switchyard AI Gateway?

Switchyard AI Gateway is an open-source LLM traffic proxy. It stays between your coding agent or application and one or more model backends. Its documented roles include protocol conversion, routing, configured fallbacks, request statistics, and profile-based flows. It is useful when you want clients to keep their native API format while your platform controls backend selection.

Is Switchyard written in Rust or Python?

The safest answer is both, but not as one undifferentiated implementation. The main proxy and CLI workflow is Python. The repository also provides an independent Rust server and shared Rust crates. You should document which path you deploy, because package installation, configuration, runtime behavior, and operational checks may differ between the Python proxy and Rust server.

How does Switchyard connect to Claude Code?

Use the documented launcher path. Provide the upstream API key, base URL, and model, then run switchyard launch claude. Switchyard starts a local proxy and configures the client to use it. After that, test plain text, streaming, tools, MCP, model aliases, and fallback behavior. A successful launcher start alone does not prove full Claude Code compatibility.

Which model protocols does Switchyard support?

The official project documents OpenAI Chat Completions, Anthropic Messages, and OpenAI Responses formats. It also supports OpenAI-compatible endpoints such as vLLM and Ollama when their exposed API matches the expected contract. Provider-specific extensions remain a separate compatibility question, especially for tools, structured output, streaming, and reasoning fields.

Can Switchyard run as a standalone proxy service?

Yes. The Python server mode can expose configured routes through an HTTP endpoint, with route names available as model IDs. That makes it suitable for a local shared proxy or an internal service. Before using it as a persistent gateway, add access control, secret handling, health checks, logs, restart supervision, session tests, and a documented upgrade procedure.

When Hashvps is the better operating environment

Running Switchyard on a laptop is fine for a first test, but it becomes awkward when the proxy must stay available for several developers or automation jobs. A local machine can sleep, change networks, lose its Python environment, or expose provider credentials through a developer shell. It also makes centralized logs, stable routing profiles, and restart handling harder to enforce.

A persistent Hashvps environment is worth considering when you need a stable host for an internal AI Gateway, repeatable Claude Code or Codex access, and a place to validate long-running request paths. You still need to choose the right setup: local deployment is simpler for short experiments, self-managed hardware is better when you need physical interfaces or sustained private workloads, and a hosted environment is usually easier for temporary or shared testing.

Before moving, confirm your protocol requirements, expected session behavior, secret policy, and Rust-versus-Python component choice. Then use the AI Gateway deployment acceptance guide to verify the host, request chain, and recovery process instead of treating “the proxy started” as the finish line.