Git.Top Guide

Open Source LLM Gateways

Model routing, provider abstraction, OpenAI-compatible proxying, fallback, policy, and cost-control projects.

Guide

What To Inspect

Compare LLM gateways by provider support, OpenAI compatibility, routing controls, fallback behavior, policy hooks, deployment target, and observability integrations.

Guide

Best Use

Use this page when choosing infrastructure between AI applications and model providers, especially when teams need routing, budget control, provider failover, or a unified API surface.

Data Source

d1 / d1_query

1067 loaded projects. Generated at 2026-08-13T13:35:33.660Z.

Cost-efficient and pluggable Infrastructure components for GenAI inference

Llm GatewayKubernetesLocalCloud
Quality32
Agent77

Project

Hal0ai/hal0

63

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + systemd.

Llm GatewayDockerLocalCloud
Quality30
Agent63

Project

off-grid-ai/OGAD

64

Off Grid AI — private, on-device AI. Run open models (text, vision, image, voice) locally through one OpenAI-compatible gateway. No cloud, no accounts, no API keys.

Llm GatewayLibrary OnlyLocalCloud
Quality30
Agent64

High-performance proxy router for LLM APIs with automatic load balancing, rate limiting, and fail2ban protection

Llm GatewayDockerLocalCloud
Quality25
Agent62

OpenAPI specification for the OpenAI API

Llm GatewayLocalCloud
Quality15
Agent67

Antigravity Proxy is a high-performance gateway that exposes Google's internal Gemini and Claude APIs through an OpenAI-compatible interface

Llm GatewayDockerLocalCloud
Quality14
Agent55

OpenAI-compatible gateway aggregating 8 free LLM providers with automatic failover, circuit breakers, and smart routing. 32+ models, zero cost.

Llm GatewayDockerCloudflareServerless
Quality13
Agent58

Zero trust LLM gateway. OpenAI-compatible proxy with semantic routing and load balancing across OpenAI, Anthropic, Ollama, vLLM, and any compatible backend. Identity-based access, virtual API keys, and end-to-end encryption via OpenZiti

Llm GatewayLocalCloud
Quality10
Agent54

Project

andreimerfu/pllm

54

High-performance LLM Gateway built in Go - OpenAI compatible proxy with multi-provider support, adaptive routing, and enterprise features

Llm GatewayDockerKubernetesLocal
Quality11
Agent54

Project

BuilderIO/ai-shell

64

A CLI that converts natural language to shell commands.

Llm GatewayLibrary OnlyLocalCloud
Quality10
Agent64

Project

Nutlope/llamacoder

62

Open source Claude Artifacts – built with Llama 3.1 405B

Llm GatewayLibrary OnlyLocalCloud
Quality8
Agent62