FreeToken
Verifiedby FlashML
Serves large Mixture-of-Experts models such as DeepSeek-V4-Flash and GLM-5.2 on consumer NVIDIA GPUs, sharing work across GPU and CPU behind OpenAI- and Anthropic-compatible APIs.
- Access
- CLIAPI
- Auth
- None
- Skills
- 3
Skills
Compatible API Server
Starts a local server with ft serve that exposes OpenAI /v1 endpoints, the Anthropic Messages API and the Responses API for a chosen model.
Coding Agent Launcher
Discovers the served model with ft launch, writes the coding agent's provider config, installs its CLI if missing and launches it.
MoE Expert Offload
Runs MoE layers with bandwidth-adaptive CPU-GPU co-execution and LRU expert caching, reallocating VRAM between experts and KV at runtime.
Related Agents
Strata
by Niko1221
Runs the Qwen3.8-Flash-Next model on a 12 GB+ consumer GPU and serves it locally over OpenAI- and Anthropic-compatible…
Baseten
by Baseten
Deploys models as production inference APIs with dedicated GPU deployments, Truss packaging, and OpenAI-compatible…
oMLX
by Jun Kim
Serves local LLM, vision, embedding and reranker models on Apple Silicon Macs through OpenAI- and Anthropic-compatible…
ds4
by Salvatore Sanfilippo (antirez)
Runs DeepSeek V4 Flash, GLM 5.x, and Qwen models locally on Metal, CUDA, or ROCm with a native engine, a CLI, a coding…