Skip to main content

Announcing the first Envoy AI Gateway Release – A Community Milestone!

· 3 min read
Erica Hughberg
Erica Hughberg
Envoy AI Gateway Maintainer - Tetrate
Dan Sun
Dan Sun
Envoy AI Gateway Maintainer - Bloomberg
Takeshi Yoneda
Takeshi Yoneda
Envoy AI Gateway Maintainer - Tetrate
Aaron Choo
Aaron Choo
Envoy AI Gateway Maintainer - Bloomberg
Yao Weng
Yao Weng
Envoy AI Gateway Maintainer - Bloomberg

Announcing the first Envoy AI Gateway Release

Today, we're excited to announce the 0.1 release of the Envoy AI Gateway, the first AI gateway built on CNCF's Envoy Gateway and backed by a thriving, growing community.

The journey to the Envoy AI Gateway started with a simple but powerful vision: make it easier for enterprises to integrate and scale AI in their applications.

Where We Are Now​

The Envoy AI Gateway is now available on GitHub and ready for developers to deploy and explore. It enables enterprises to integrate AI services through a unified API while managing authorization, cost control, and scalability with built-in features:

  • ✅ Unified API for seamless integration with multiple LLM providers (starting with AWS Bedrock and OpenAI).
  • ✅ Upstream Authorization to simplify authentication across AI providers.
  • ✅ Usage Rate Limiting based on word tokens to control costs and ensure operational efficiency.

With this release, we're making AI adoption in cloud-native environments more straightforward and accessible to organizations of all sizes.

The Power of Community​

This milestone wouldn't be possible without the incredible contributions and participation from across the industry.

A shout out to our community members from Tetrate, Bloomberg, WSO2, RedHat, Google, and our independent contributors who have joined discussions, provided feedback, and helped shape the roadmap.

🐱 Even the cat Mellow has attended community meetings, proving that AI isn't just for humans!

The momentum behind Envoy AI Gateway speaks to the need for an open, collaborative approach to GenAI infrastructure. Our contributor's and early adopters' excitement and expertise drive innovation forward.

Where We're Headed​

We're just getting started! Here's a sneak peek at what's next:

  • Google Gemini 2.0 integration out-of-the-box.
  • Provider and Model Fallback Logic to ensure continued service availability.
  • Prompt Templating for consistent AI interactions.
  • Semantic Caching to optimize response efficiency and reduce costs.

The roadmap is community-driven, and we'd love for more contributors to help shape the future of AI infrastructure!

Join the Movement​

Want to be part of the journey? Here's how you can get involved:

The future of GenAI infrastructure is open and collaborative, and we're excited to work with you to build it!

🚀 Onward to 1.0!