Xunzhuo (Bit) Liu

The Envoy AI Gateway v0.3 release introduces intelligent inference routing through Endpoint Picker (EPP) integration, expands our provider ecosystem with Google Vertex AI Production Support as well as Native Anthropic API, and delivers Enterprise-Grade Observability with OpenInference tracing.

The Big Shifts in v0.3​

Envoy AI Gateway v0.3 isn't just another feature release; it's a fundamental shift toward intelligent, production-ready AI infrastructure. This release addresses three critical challenges that have been holding back AI adoption in enterprise environments:

Introduction​

Envoy AI Gateway now supports Endpoint Picker Provider (EPP) integration as per the Gateway API Inference Extension.

This feature enables you to leverage intelligent, dynamic routing for AI inference workloads through intelligent endpoint selection based on real-time metrics, including KV-cache usage, queued requests, and LoRA adapter information.

When running AI inference at scale, this means your system can automatically select the optimal inference endpoint for each request, thereby optimizing resource utilization.