Real-time multi-model routing — see how intelligent routing achieves cost and latency savings
Per-model cost values shown below are illustrative estimates from the configured token-rate table in `/api/llm-router`, used for relative comparison only.
Route intent: prefer lower-cost low-latency tiers for simple queries, then escalate to higher-capability models only when prompt complexity requires it.
Mid-tier routes balance coding quality, latency, and single-GPU deployment constraints.
Qwen 3.8 27B
mid tier · dense
Dense 27B open-weight model (Apache 2.0) with thinking preserved across turns by default; Qwen self-reports 61.7 on SWE-bench Pro. Quantized builds (GGUF) are linked from the model card for local use via llama.cpp, Ollama or LM Studio, and the model is also served on Groq (preview).
VRAM: ~17GB (Q4, approximate)
Context: 131,072 tokens
Qwen3.6-35B-A3B-int4
mid tier · moe
Mixture-of-experts model tuned for low-latency UI paths where only a small active parameter slice is used per token.
VRAM: ~21GB (Q4)
Context: 32,768 tokens
Qwen3.8-27B Routing Rule
task.type === "agent" || task.complexity === "medium-high"
Dense architecture + Thinking Preservation = stable multi-turn reasoning without MoE routing overhead.
Local fallback: when external API unavailable → Qwen3.8-27B GGUF via llama.cpp
| Dense vs MoE | 27B Dense | 35B-A3B MoE |
|---|---|---|
| Active params | 27B | 3B |
| Inference speed | Moderate | Fast |
| VRAM (Q4) | ~16.8GB | ~21GB |
| Deployment | llama.cpp / LM Studio (GGUF) | API/Cloud |
| Best for | Agent loops | Low-latency UI |