Before moving on, please consider giving us a GitHub star ⭐️. Thank you!
Zilla is a streaming-native, multi-protocol gateway for AI, APIs, and real-time data.
Connect applications and agents to LLM providers, enterprise tools, and event streams through one high-performance gateway. Zilla handles protocol translation, routing, security, and observability at the edge, without requiring separate proxy services or custom integration code for supported backends.
Use existing AI SDKs to access models across providers. Give agents one governed MCP endpoint for enterprise tools and data. Expose Kafka streams through application-friendly protocols.
Zilla composes protocol-native bindings in a lightweight, stateless runtime, configured declaratively in zilla.yaml. LLM access, MCP tool connectivity, and event streaming are different uses of the same gateway architecture, not three separately deployed gateways.
Contents
What You Can Build
Connect applications to LLMs
Use OpenAI, Anthropic, or Google Gemini SDKs with Zilla as the gateway endpoint. Route requests by model, translate supported chat and tool-call workflows between provider APIs, stream responses, apply configurable prompt and response guardrails, and export token and latency metrics. Configure fallback when a model deployment cannot be reached. Zilla Plus adds Amazon Bedrock, including access from existing SDKs.
Try it: Cross-provider LLM demo | LLM documentation
Aggregate existing MCP servers, turn HTTP/OpenAPI services into tools, and expose Kafka capabilities directly over MCP without building wrapper services for those backends. Present a single namespaced MCP endpoint, authorize what each caller can discover and invoke, and keep the default tool list compact with on-demand discovery.
Try it: MCP example | MCP documentation
Bring real-time data to applications, services, and devices
Expose Kafka-backed REST APIs and live streams to HTTP, SSE, WebSocket, gRPC, and MQTT clients. Apply protocol-aware authentication, authorization, schema validation, and data policies at the gateway instead of building and operating custom bridges.
Try it: Kafka REST example | Kafka Gateway documentation
Get Started
The examples below use Docker Compose. Run them in separate sessions: several examples share local ports.
1. Try the LLM gateway
Prerequisites: Docker Compose and API keys for the model providers you want to call.
git clone --depth 1 https://github.com/aklivity/zilla-demos.git
cd zilla-demos/llm.gateway
cp .env.example .env
Edit .env to add your provider keys:
ZILLA_VERSION=2.6.0
OPENAI_API_KEY=<your-openai-api-key>
ANTHROPIC_API_KEY=<your-anthropic-api-key>
GEMINI_API_KEY=<your-gemini-api-key>
For Apple silicon or another ARM64 host, also uncomment TEI_VERSION=cpu-arm64-1.9 in .env. Keep .env out of version control.
docker compose up --build --wait
Open the demo app at localhost:7860. For example, select the OpenAI SDK, choose a claude-* model, and send a prompt. Zilla routes it to Anthropic, translates the request and response, and returns the result in the OpenAI format your SDK expects.
Or try the same cross-provider call from the command line:
docker compose run --rm -e SDK=openai -e MODEL=claude-haiku-4-5 llm-client
Explore streaming replies, tool calls, guardrail rejections, model fallback, and comparisons across providers. Open the Grafana dashboard for token usage, request counts, latency, and time to first token, or inspect raw metrics:
curl -s http://localhost:7190/metrics | grep '^llm_'
The demo uses bring-your-own-key credentials: callers send a valid key for the provider handling the request. Translating an API does not make a key from one provider valid at another. Gateway-managed credentials can also be configured. Some guardrail checks in this demo use accompanying Presidio and embedding services.
For the full walkthrough, including Gemini and the Zilla Plus / Bedrock extension, see the LLM demo README.
2. Try the MCP gateway
From a separate terminal, after stopping the previous demo (docker compose down), run:
git clone --depth 1 --branch 2.6.0 https://github.com/aklivity/zilla.git
cd zilla/examples/mcp.proxy
docker compose up --wait
The gateway listens at http://localhost:7114/mcp over MCP Streamable HTTP. This example combines MCP servers, HTTP/OpenAPI-backed tools, and Kafka-backed tools. Its protected toolkits require JWT scopes; use the instructions in the example README to obtain a token and connect Claude Code or MCP Inspector.
3. Expose Kafka through a REST API
After stopping the MCP example (docker compose down), from the cloned zilla repository:
cd ../http.kafka.crud
docker compose up --wait
curl -X POST http://localhost:7114/items \
-H 'Content-Type: application/json' \
-d '{"greeting":"Hello from Zilla"}'
curl http://localhost:7114/items
This example backs REST create/read/update/delete operations with a Kafka topic. Browse the records in Kafka UI.
Browse all examples | Read the documentation
Why Zilla?
Most systems do not speak the protocols of the services they need. AI applications call model APIs; agents use MCP; business services expose HTTP and gRPC; real-time data lives in Kafka and MQTT. Building bridges between them typically means more adapters, proxies, credentials, and operational overhead.
Zilla addresses that fragmentation at the gateway:
- One runtime for many protocols. Compose LLM, MCP, HTTP, SSE, WebSocket, gRPC, Kafka, and MQTT connectivity in a single deployment, with shared configuration and telemetry infrastructure.
- Native translation instead of application-side wrappers. Connect supported models, APIs, tools, and event streams through protocol-aware bindings, rather than maintaining custom middleware for every integration.
- Governance at the boundary. Enforce caller identity, route and model access, MCP tool visibility and invocation, schema rules, and configured LLM guardrails independently of client implementations. Controls remain specific to each protocol and policy configuration.
- Streaming-native performance. Preserve backpressure and avoid unnecessary buffering, copies, and allocations along the data path, including streaming LLM responses and event-driven traffic.
- Simple deployment and operations. Run a lightweight, stateless data plane in Docker or Kubernetes. External stores are optional for features that need shared state across replicas.
How It Works
Zilla connects protocol-aware bindings into a route. A binding terminates or originates a protocol; routing and configured guards determine where traffic goes and what is permitted. LLMs, MCP tools, and event streams use the same streaming engine and operational model.
Under the hood, code-generated flyweights access encoded buffers without unnecessary object creation. Connections stay assigned to a worker to reduce contention, and bindings exchange back-pressured stream frames through the runtime. Stateless processing enables horizontal scaling; shared stores can be configured where distributed features require them.
How Zilla Works | Configuration reference
Capabilities
| Area |
What Zilla enables |
| Protocol connectivity |
HTTP, SSE, WebSocket, gRPC, MQTT, Kafka, MCP, and supported LLM APIs in one composable runtime |
| LLM providers and APIs |
OpenAI Chat Completions, Anthropic Messages, and Google Gemini in Zilla Community; Amazon Bedrock in Zilla Plus |
| LLM routing and interoperability |
Model-based routing, supported request/response and tool-call translation, streaming, and connectivity-based fallback |
| LLM governance |
Caller authentication, model-aware authorization, configurable prompt/response guardrails, and provider credential handling |
| MCP connectivity |
Aggregate MCP servers; expose HTTP/OpenAPI and Kafka-backed capabilities as tools without custom wrappers for supported backends |
| MCP governance and discovery |
Identity-aware tool listing and invocation; namespaced toolkits; eager and on-demand discovery |
| API and event integration |
Expose Kafka through REST and streaming protocols; connect services and devices across supported protocols |
| Data validation |
Protocol-appropriate schema validation and supported JSON, Avro, and Protobuf transformations |
| Observability |
Gateway and protocol metrics, MCP interaction metrics, and provider-reported LLM token usage and latency; Prometheus/OTLP export |
| Deployment |
Docker and Kubernetes/Helm; stateless data-plane scaling, with optional external shared stores where needed |
LLM compatibility notes: Translation covers supported chat, streaming, and tool workflows, not every provider-specific parameter or API feature. Configured fallback is for deployments that cannot be reached; it does not trigger on provider responses such as HTTP 429 or 5xx. See dialect translation and fallback.
Explore the Documentation
Zilla Plus
Zilla Plus builds on Zilla Community with additional capabilities for enterprise Kafka and AI deployments:
- Native Kafka-to-Kafka proxying. Transparently connect existing Kafka producers, consumers, and Kafka Connect clients to Kafka clusters using the native Kafka protocol, without application changes.
- Virtual Kafka clusters. Give teams and tenants isolated Kafka endpoints, topic and consumer-group namespaces, and access controls on shared Kafka infrastructure.
- Dynamic topic aliasing. Map the same client-facing topic name to different internal topics based on client identity.
- Secure public and private Kafka access. Connect clients through authenticated, TLS-protected public endpoints or private connectivity across VPCs and AWS accounts, including Amazon MSK and MSK Serverless.
- Additional enterprise governance and integrations. Field-level data protection, Amazon Bedrock integration, advanced OAuth capabilities, distributed stores, and commercial support.
Explore the Kafka Gateway use cases and edition comparison for details.
Get started with Zilla Plus: Request a license key from the Aklivity team or get Zilla Plus on AWS Marketplace.
Install with Your Own Configuration
For a guided first experience, use the LLM quickstart above. The commands below run Zilla with a configuration you supply; they do not generate a working zilla.yaml for you.
Docker
docker pull ghcr.io/aklivity/zilla:2.6.0
docker run --rm \
-p 7114:7114 \
-v "$(pwd)/zilla.yaml:/etc/zilla/zilla.yaml:ro" \
ghcr.io/aklivity/zilla:2.6.0 \
start -v
Adjust published ports to match the listeners in your configuration.
Kubernetes / Helm
helm install zilla oci://ghcr.io/aklivity/charts/zilla \
--namespace zilla \
--create-namespace \
--wait \
--values values.yaml \
--set-file 'zilla\.yaml=zilla.yaml'
Choose chart settings and gateway image version appropriate for your deployment. See the deployment documentation.
Resources
License
Zilla is made available under the Aklivity Community License.
The license allows you to deploy, run, and modify Zilla for your own workloads, including production and cloud deployments. It does not permit offering Zilla as a standalone commercial Zilla-as-a-service product. Review the license text for the full terms.
(Back to top)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?v=4)
![dependabot[bot]](https://avatars.githubusercontent.com/in/29110?v=4)
评论