Local Models
Per-model adaptations live in platforms/local-models/. General laws first — they matter more than any per-model trick:
- One task spec per fresh context (small models degrade fastest with context length)
- Force the output format; reject-and-retry outputs missing the footer
- External verification always — smaller model = more confident fabrication
- Right-size the role: small models are good executors and decent critics, poor planners
- More parameters at Q4 beats fewer at Q8; a wrong chat template costs more than quantization
- Tracked plans follow the same
/workcontract when the harness has shell - Docs describe current state, not plans — never write product docs from
.plans/contents
The lineup
Each model name links to its official quick start (download / serve / chat template). Use that first if you are new to the model; then apply Anchor’s adaptation notes under platforms/local-models/.
| Model | Key quirk | Role fit | Official quick start |
|---|---|---|---|
| Qwen3 (4B–32B, 30B-A3B) | /think / /no_think per message; never greedy in thinking mode | executor + critic; 32B a passable planner | Qwen3 Quickstart |
| Gemma 3 (12B/27B) | no true system role — fold instructions into first user turn; one task per conversation | obedient executor; add the BLOCKED guardrail (it won't push back on bad specs) | Gemma docs |
| Mistral Small 3.x | wants LOW temp (0.15); terse — enforce the footer in the pipeline | fast executor, best local tool-caller per GB | HF model card |
| DeepSeek-R1 distills | NO system prompt; temp ~0.6 (greedy breaks it); no few-shot; strip <think> downstream | critic + hard single problems; never an executor | DeepSeek-R1 collection |
| Llama 3.3 70B | none — the boring reliable one | generalist executor+critic if you have ~40GB VRAM | HF model card |
The quirks are encoded in scripts/endpoints.yaml (quirks: block) and applied automatically by anchor_client.py, so orchestration code never special-cases models. Hardware-specific serve helpers: personal devices, H100.
Tracked plans and coordination
Local models follow the same /work contract when the harness has shell: honor Preferred models and Depends on, claim into in-progress/, never promote drafts. They do not self-appoint as temporary coordinator when Preferred orchestrator is unset—escalate to a frontier session. See model fitness and Skills → /work.