Templates cover Plan / implement / review, Code / writing / analysis, Quick / thorough, Client mode, and a custom question. Adjust descriptions to resolve overlapping categories. The user chooses which model handles each route.

For an exact plan-mode rule, have your client send metadata: {"localrouter.mode": "plan"}. Map plan to your planning route and set the default to your other model. Missing or unrecognized mode uses the default. Prompt text cannot override this metadata. OpenAI does not provide a universal plan/edit inference field, so the client must supply this LocalRouter convention.

Semantic routing receives the latest user request and bounded recent history with roles preserved. Set a timeout, character budget, history count and optional minimum probability. If the latest request exceeds the budget or a provider rejects the input, use the default route. Character budgets are not exact tokenizer limits; preview with your selected model.

Existing strong/weak configurations migrate their model lists without retaining the old threshold. Choose a decision provider to enable semantic classification; until then the previous strong/default list is used.

See our local experiments for measured behavior and limitations.

Provider failover is automatic and based on the type of error received:

• Rate limited — The provider is at capacity. The router moves to the next model immediately.
• Policy violation — Content was rejected by the provider's safety filters. The router tries the next model, which may have different content policies.
• Context length exceeded — The input is too long for this model. The router tries the next model, which may have a larger context window.
• Unreachable — The provider is down or not responding. The router tries the next model.
• Other errors (e.g., validation failures) — These are not retryable and cause an immediate error response.

If the strategy's prioritized model list includes local providers (Ollama, LM Studio) alongside remote providers, the router automatically falls through to local models when remote providers are unreachable. No special configuration is needed — just include local models in your fallback chain.

Order the prioritized models list from cheapest to most expensive. When a request hits localrouter/auto, the cheapest available model is always tried first. A custom policy can assign routine tasks to a cheaper model, with your chosen default and fallback lists.

Place the most capable models first in the prioritized list. The top-tier model handles every request unless it fails, in which case the router falls back to the next best model. A custom policy can reserve particular models for planning, review or other work you define.

Place local providers (Ollama, LM Studio) at the top of the prioritized list with remote providers as fallbacks. Requests route to locally-running models by default, avoiding API costs and network latency. If the local provider is down, the router falls through to remote providers.

Place cloud providers (OpenAI, Anthropic, etc.) at the top of the prioritized list with local models as fallbacks. This prioritizes the quality and speed of remote API models while maintaining resilience — if the remote provider is rate-limited or unreachable, requests automatically fail over to local models.

Each provider is assigned one of six free-tier types. The type determines how the router tracks usage and decides whether the provider still has free capacity.

No Free Tier — The provider has no free API access. Always skipped in free-tier mode. Default for: OpenAI, Anthropic.

Always Free (Local) — A local or self-hosted provider with no external billing. Always treated as free, except a model given a price on the provider's Models tab. Default for: Ollama, LM Studio, and OpenAI Compatible providers whose base URL is on this machine (localhost); a remote OpenAI Compatible provider has no free tier until you set one.

Subscription — Access is included in an existing subscription plan. Always treated as free. Default for: GitHub Copilot.

Rate Limited — Free access within rate limits imposed by the provider. The router tracks usage against configurable limits:

• Requests per Minute (RPM) — Resets every minute.
• Requests per Day (RPD) — Resets at midnight.
• Tokens per Minute (TPM) — Input + output tokens per minute.
• Tokens per Day (TPD) — Tokens per calendar day.
• Monthly Call Limit — Total API calls per month. Used by Cohere (1,000 calls/month).
• Monthly Token Limit — Total tokens per month. Used by Mistral (1B tokens/month).

When any tracked limit is reached, the provider is skipped. Default for: Gemini, Groq, Cerebras, Mistral, Cohere.

Credit Based — Dollar-budget credits consumed per request. The router estimates cost from token usage and compares against the configured budget.

• Credit Budget (USD) — Total free credit allowance.
• Reset Period — When the budget resets: Daily, Monthly, or One-time (for promotional credits).

For providers that expose a balance API, the router can sync the actual remaining balance directly. Default for: xAI, DeepInfra, Perplexity.

Free Models Only — Only some models from this provider are free. Each model is decided in this order:

1. Never free — model IDs or globs (* matches anything) that are never free.
2. Always free — model IDs or globs that are always free.
3. Rules — the first enabled rule that matches decides. A rule makes matching models free, or not free, when:
   • Model ID — the ID matches exactly, a glob, or a regular expression (optionally ignoring case).
   • Listing field — a field of the model's entry in the provider's model list, addressed by a JSON Pointer such as /tier, equals, contains, matches, or compares with a value.
   • JSONPath filter — an RFC 9535 filter expression where @ is the model's entry, e.g. @.tier == 'free'.
   • $0 price — input and output price are $0 in the price set on the Models tab and/or the provider's model list.
4. Otherwise the model is not free.

Missing data never makes a model free: a model without an entry in the provider's model list, or without a known price, does not match rules that need one, and an exclude rule applies to it. Catalog prices are not used. The Free Tier tab shows a live preview of every model with why it is or is not free, lets you set a model to Always free / Never free, and opens a model's model-list entry to build a rule from one of its fields. Requests per Minute (RPM) limits requests to the free models only.

Default for: OpenRouter (models listed at $0, such as the :free variants), Together AI.

Every provider has a default free-tier type based on the provider kind. You can override this default from the provider's Free Tier tab.

Common override scenarios:

• Reclassify a provider — Change an OpenAI-compatible provider from "Always Free (Local)" to "Rate Limited" if your endpoint has rate limits.
• Adjust limits — A provider changed their free tier limits and the defaults are outdated.
• Add a free tier — Your organization has a custom agreement with a provider that normally has no free tier.
• Remove a free tier — You don't want a provider used in free-tier mode even though it has one.

Use Reset to Default to remove the override.

The router tracks free-tier usage locally based on token counts from API responses. Since this is an estimate, it can drift from your actual usage.

Set Usage lets you manually adjust the tracked counters to match reality. Open it from the Usage & Status section of a provider's Free Tier tab.

For credit-based providers:

• Credits Used (USD) — Sets how much of the budget has been consumed this period.
• Credits Remaining (USD) — Sets the remaining balance directly (e.g., from your provider dashboard). This overrides the calculated estimate.

For rate-limited providers:

• Daily Requests Used — Requests counted toward the daily limit.
• Monthly Requests Used — Requests counted toward the monthly cap.
• Monthly Tokens Used — Tokens counted toward the monthly cap.

Reset Usage clears all counters to zero and clears any active backoff state.

Free-tier usage is tracked automatically in the background. For rate-limited providers, counters reset at the appropriate intervals (per-minute, daily, monthly). When a provider returns rate limit headers in its response, those values take precedence over local counters for more accurate tracking.

For credit-based providers, cost is estimated from token usage. Providers that expose a balance API (like OpenRouter) can sync their actual remaining balance directly.

When a provider returns a rate limit (429) or credits exhausted (402) error, the router records a backoff for that provider. Subsequent requests skip backed-off providers instantly, avoiding wasted round-trips.

Backoff duration is determined from the provider's response headers when available, or uses automatic exponential backoff. A successful request clears the backoff immediately.

When all free-tier providers are exhausted, the router returns a 429 error with a Retry-After header indicating when to try again.

When a provider returns a rate limit error, the router immediately advances to the next model in the fallback chain rather than waiting for a cooldown.

Content policy violations are retryable — the router tries the next model, which may have different content policies. This allows requests rejected by one provider's safety filters to potentially succeed with a different provider.

When a provider rejects a request because the input exceeds its context window, the router tries the next model in the fallback chain, which may support a larger context window.