Git.Top Guide

Open Source LLM Gateways

Model routing, provider abstraction, OpenAI-compatible proxying, fallback, policy, and cost-control projects.

Guide

What To Inspect

Compare LLM gateways by provider support, OpenAI compatibility, routing controls, fallback behavior, policy hooks, deployment target, and observability integrations.

Guide

Best Use

Use this page when choosing infrastructure between AI applications and model providers, especially when teams need routing, budget control, provider failover, or a unified API surface.

Data Source

d1 / d1_query

1215 loaded projects. Generated at 2026-09-04T14:40:02.421Z.

OpenAI-compatible gateway for FreeBuff coding models. Token pool, session lifecycle, TLS stealth, embedded admin dashboard. No ads, no CLI, just /v1/chat/completions.

Llm GatewayDockerLibrary OnlyLocal
Quality33
Agent68

Project

off-grid-ai/OGAD

65

Off Grid AI — private, on-device AI. Run open models (text, vision, image, voice) locally through one OpenAI-compatible gateway. No cloud, no accounts, no API keys.

Llm GatewayLibrary OnlyLocalCloud
Quality31
Agent65

Project

Hal0ai/hal0

62

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + systemd.

Llm GatewayLocalCloud
Quality30
Agent62

Project

VkRainB/ccMesh

64

Claude Code / Codex local AI proxy gateway desktop app (Tauri). Anthropic Messages <-> OpenAI-compatible protocol transform, model mapping, endpoint rotation/failover. 本地桌面代理网关:协议转换、模型映射、多端点轮换/熔断,对接官方 API 或中转站,不改客户端。Windows/macOS/Linux.

Llm GatewayLibrary OnlyLocalCloud
Quality30
Agent64

Cost-efficient and pluggable Infrastructure components for GenAI inference

Llm GatewayKubernetesLocalCloud
Quality29
Agent77

Project

sxueck/llm-gateway

61

Lightweight distributed LLM gateway w/ web UI for model mgmt & routing. Supports vibe programming, prompt opt., & optimized OpenAI API/Anthropic calls

Llm GatewayDockerLocalCloud
Quality23
Agent61

High-performance proxy router for LLM APIs with automatic load balancing, rate limiting, and fail2ban protection

Llm GatewayDockerLocalCloud
Quality23
Agent61

Project

Ahoo-Wang/GodeX

57

Make every model a CodeX engine through an OpenAI-compatible Responses API gateway

Llm GatewayLocalCloud
Quality23
Agent57

OpenAPI specification for the OpenAI API

Llm GatewayLocalCloud
Quality17
Agent67

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

Llm GatewayLocalCloud
Quality13
Agent55

OpenAI-compatible gateway aggregating 8 free LLM providers with automatic failover, circuit breakers, and smart routing. 32+ models, zero cost.

Llm GatewayDockerCloudflareServerless
Quality13
Agent59

Project

BuilderIO/ai-shell

64

A CLI that converts natural language to shell commands.

Llm GatewayLibrary OnlyLocalCloud
Quality10
Agent64