Free and easy access to large language models — for everyone.
Providing access to AI by intelligently routing requests across free tiers of LLM providers.
FreeLLM is an open-source proxy that lets you use large language models completely free by leveraging the free tiers offered by various LLM providers. It tracks usage in real time and automatically routes requests to models that still have available quota — so you never hit a rate limit wall.
- Automatic model rotation — Tracks token usage, requests per minute, and daily quotas. When one model's limits are reached, requests are seamlessly routed to the next available model.
- OpenAI-compatible API — FreeLLM exposes standard OpenAI-compatible endpoints. Any tool, library, or agent that speaks the OpenAI API can connect directly.
- Coding agent ready — Works out of the box with coding agents like PI. PI is the recommended agent, but any OpenAI-compatible client will work.
- Multi-provider support — Configure multiple LLM providers and models. FreeLLM maximizes your combined free-tier capacity.
| Provider | Model | Tokens/min | Requests/min | Requests/day |
|---|---|---|---|---|
| Gemini | gemini-2.5-flash | 250,000 | 5 | 20 |
| Gemini | gemini-3-flash | 250,000 | 5 | 20 |
| Gemini | gemini-3.5-flash | 250,000 | 5 | 20 |
| Gemini | gemini-3.1-flash-lite | 250,000 | 15 | 500 |
| Gemini | gemini-2.5-flash-lite | 250,000 | 10 | 20 |
| Gemini | gemma-4-26b | unlimited | 15 | 1,500 |
| Gemini | gemma-4-31b | unlimited | 15 | 1,500 |
| Groq | llama-3.1-8b-instant | 6,000 | 30 | 14,400 |
| Groq | llama-3.3-70b-versatile | 12,000 | 30 | 1,000 |
Combined capacity: 1,268,000 tokens/min • 130 requests/min • 18,980 requests/day
The fastest way to get FreeLLM running is with the pre-built Docker image from Docker Hub.
curl -L https://github.com/CezaryChodun/FreeLLM/archive/refs/heads/main.zip -o /tmp/freellm.zip && \
unzip /tmp/freellm.zip 'FreeLLM-main/examples/docker-compose/*' -d /tmp/freellm_extract && \
cp -r /tmp/freellm_extract/FreeLLM-main/examples/docker-compose FreeLLM && \
rm -rf /tmp/freellm.zip /tmp/freellm_extract
cd FreeLLMcp .env.example .env- Set
LITELLM_SALT_KEY,LITELLM_MASTER_KEY,LITELLM_DB_PASSWORD, and API keys: - Gemini API key
- Groq API key
Important! Use only free API keys for LLM access. While FreeLLM keeps track of usage you might incure cost if your key is assigned to a paid plan.
# Start all services
docker compose up -dFreeLLM will be available at http://localhost:3000. Point any OpenAI-compatible client at this URL.
Note:
- FreeLLM doesn't store or use your keys. They are used by LiteLLM under the hood
- For non-local deployments, change the default database password in
docker-compose.ymlto a strong, unique password.
The example spins up four containers:
- freellm — the proxy (port 3000)
- litellm — LLM gateway (port 4000)
- postgres — usage tracking database
- prometheus — metrics collection (port 9090)
- Choose models — edit
config.yml - Adjust rate limits — edit
defaults/gemini.yml - Add providers — add new entries to
litellm-config.yaml,config.yml, and corresponding defaults
All sensitive configuration is provided via .env. Copy .env.example and fill in your values:
| Variable | Description |
|---|---|
LITELLM_MASTER_KEY |
Admin key for LiteLLM API access |
LITELLM_SALT_KEY |
Encryption key for stored API keys |
LITELLM_DB_PASSWORD |
Password for the LiteLLM PostgreSQL user |
GEMINI_API_KEY |
Your Google AI API key for Gemini/Gemma models |
GEMINI_API_BASE |
Google AI API base URL |
GROQ_API_KEY |
Your Groq API key for Llama/other Groq models |
Models are defined in config.yml with a provider/model format:
models:
- model: gemini/gemini-2.5-flash
- model: gemini/gemma-3-27b-itRate limits for each provider are stored in the defaults/ directory:
# defaults/gemini.yml
- name: gemini-2.5-flash
TPM: 250000
RPM: 5
RPD: 20FreeLLM loads these at startup, populates the rate limits database, and begins routing immediately.
Model groups let you organize models into named sets and route requests to a specific subset of your configured models. When a client sends a request with a group name as the model field, FreeLLM only considers models within that group for routing. If no matching group is found, all models are considered.
Add a model_groups section to your config.yml. Each group has a name and a list of models (using the same provider/model format):
models:
- model: gemini/gemini-2.5-flash
- model: gemini/gemini-3-flash
- model: gemini/gemini-3.1-flash-lite
- model: gemini/gemma-4-26b
- model: groq/llama-3.3-70b-versatile
model_groups:
- name: flash
models:
- gemini/gemini-2.5-flash
- gemini/gemini-3-flash
- gemini/gemini-3.1-flash-lite
- name: large
models:
- gemini/gemma-4-26b
- groq/llama-3.3-70b-versatileModels referenced in a group must also be listed in the top-level models section.
Set the model field in your API request to the group name. FreeLLM will select the best available model within that group:
curl http://localhost:3000/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "flash",
"messages": [{"role": "user", "content": "Hello"}]
}'The /v1/models endpoint lists all available groups alongside a special all entry that routes across every configured model:
curl http://localhost:3000/v1/modelsWithin a group (or across all models when no group is specified), FreeLLM routes each request to the model that is furthest from hitting its rate limits.
- Compute utilization per metric — For each candidate model, FreeLLM calculates the current usage as a percentage of the configured limit for each metric (tokens/min, requests/min, requests/day).
- Most constrained metric wins — The highest utilization percentage across all metrics becomes the model's overall utilization score. This means a model that is close to exhausting any single limit is considered highly utilized, even if other metrics have plenty of headroom.
- Pick the least utilized model — The model with the lowest utilization score is selected, spreading load evenly across available capacity.
- Exhausted models are excluded — Any model that has fully reached its limit on any metric is removed from consideration entirely.
Given two models in a group:
| Model | Tokens/min used | Tokens/min limit | RPM used | RPM limit | RPD used | RPD limit |
|---|---|---|---|---|---|---|
| gemini-2.5-flash | 50,000 | 250,000 (20%) | 3 | 5 (60%) | 10 | 20 (50%) |
| gemini-3-flash | 100,000 | 250,000 (40%) | 1 | 5 (20%) | 5 | 20 (25%) |
- gemini-2.5-flash utilization = max(20%, 60%, 50%) = 60%
- gemini-3-flash utilization = max(40%, 20%, 25%) = 40%
FreeLLM routes the request to gemini-3-flash because it has the lower utilization score.
┌─────────────┐ ┌──────────┐ ┌──────────┐ ┌──────────────┐
│ Your App │────▶│ FreeLLM │────▶│ LiteLLM │────▶│ LLM Provider │
│ (PI, etc.) │ │ (proxy) │ │ │ │ (Gemini...) │
└─────────────┘ └──────────┘ └──────────┘ └──────────────┘
FreeLLM sits as a proxy layer on top of LiteLLM. LiteLLM handles the connection management and provider-specific API translations. FreeLLM adds:
- Usage tracking — Persists per-model token and request counters in PostgreSQL
- Rate limit awareness — Compares real-time usage against configured quotas
- Intelligent routing — Model selection filtered to only models with remaining capacity
This project is licensed under the GNU General Public License v3.0 — see the LICENSE file for details. Learn more about GPL v3.0.