BACK TO SELECTED WORK
CASE STUDY 02 / 09Systems / Infrastructure
OpenAI-compatible LLM infrastructure

Indus LLM Gateway

A production-ready, self-hosted LLM gateway that unifies provider routing, API authentication, quotas, memory, streaming, metrics, and dashboard visibility behind one OpenAI-compatible endpoint.

GoOpenAI APISSEPrometheusYAMLWindows EXE
Indus LLM Gateway project previewPROJECT / CINEMATIC
01 / THE WORK

Behind the build.

Indus LLM Gateway is built for teams that want one controlled entry point for AI traffic without rewriting their application code. It acts as a reverse proxy for OpenAI-compatible APIs, routes requests across providers and models, enforces API-key budgets, proxies streaming responses, injects prompt templates, tracks cost, and exposes Prometheus metrics plus an embedded web dashboard.

02 / ENGINEERING DECISIONS

What makes it work.

01 / CAPABILITYOpenAI-compatible API surface for drop-in SDK usage
02 / CAPABILITYMulti-provider and multi-model routing with failover-minded control
03 / CAPABILITYAPI keys, quotas, token accounting, and usage cost tracking
04 / CAPABILITYEmbedded dashboard and Prometheus metrics for operational visibility
03 / INTERACTIVE WALKTHROUGH

From input to outcome.

STAGE 1 OF 5

Applications send standard chat completion requests to the gateway.

04 / ARCHITECTURE

Under the hood.

Reverse Proxy Topology & Governance Loop

Architecture Diagram 2.1
Step 1 — IngestionOpenAI SDK Clients

Standard `/v1/chat/completions` REST/SSE requests sent to Gateway endpoint.

Step 2 — MiddlewareAuth & Quota Policy

Bearer token validation, rate limiters, token accounting, and budget enforcement.

Step 3 — Smart RoutingProvider Failover

Dynamic model dispatcher across OpenAI, Anthropic, Gemini, or local models.

Step 4 — ObservabilityPrometheus & Dashboard

Cost estimation, latency histograms, token usage, and live dashboard graphs.

05 / RESULTS & USE CASES

Built to matter.

OUTCOME 01Centralized governance for LLM usage
OUTCOME 02Provider flexibility without client-side rewrites
OUTCOME 03Production visibility into tokens, latency, cost, and health
AI product teamsInternal platformsLLM cost controlProvider abstraction