Skip to main content
Version: latest

Connect Tetrate Agent Router Service (TARS)

This guide will help you configure Envoy AI Gateway to work with Tetrate Agent Router Service (TARS)'s models.

Prerequisites​

Before you begin, you'll need:

Configuration Steps​

Ready to proceed?

Ensure you have followed the steps in Connect Providers

1. Download configuration template​

curl -O https://raw.githubusercontent.com/envoyproxy/ai-gateway/main/examples/basic/tars.yaml

2. Configure Tetrate Agent Router Service (TARS) Credentials​

Edit the tars.yaml file to replace the TARS placeholder value:

  • Find the section containing TARS_API_KEY
  • Replace it with your actual TARS API key
Security Note

Make sure to keep your API key secure and never commit it to version control. The key will be stored in a Kubernetes secret.

3. Apply Configuration​

Apply the updated configuration and wait for the Gateway pod to be ready. If you already have a Gateway running, then the secret credential update will be picked up automatically in a few seconds.

kubectl apply -f tars.yaml

kubectl wait pods --timeout=2m \
  -l gateway.envoyproxy.io/owning-gateway-name=envoy-ai-gateway-basic \
  -n envoy-gateway-system \
  --for=condition=Ready

4. Test the Configuration​

You should have set $GATEWAY_URL as part of the basic setup before connecting to providers. See the Basic Usage page for instructions.

curl -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o-mini",
    "messages": [
      {
        "role": "user",
        "content": "Hi."
      }
    ]
  }' \
  $GATEWAY_URL/v1/chat/completions

Troubleshooting​

If you encounter issues:

  1. Verify your API key is correct and active

  2. Check pod status:

    kubectl get pods -n envoy-gateway-system

  3. View controller logs:

    kubectl logs -n envoy-ai-gateway-system deployment/ai-gateway-controller

  4. View External Processor Logs

    kubectl logs -n envoy-gateway-system -l gateway.envoyproxy.io/owning-gateway-name=envoy-ai-gateway-basic -c ai-gateway-extproc

  5. Common errors:

    • 401: Invalid API key
    • 429: Rate limit exceeded
    • 503: TARS service unavailable