How to Run Saluki 27B Locally: 2-Bit Qwen3-27B Tool-Calling at Minimal Compute Cost

A practical guide covers running Saluki 27B — a 2-bit quantized derivative of Qwen3-27B optimized for tool-calling — using llama.cpp on consumer or modest server hardware. The model targets the efficiency-capability tradeoff for developers who need reliable function-calling and structured output at 27B-parameter quality without GPU-heavy inference infrastructure. The guide walks through model loading, quantization tradeoffs, and tool-calling configuration specific to this model's architecture. For developers building local-first agent pipelines or prototyping tool-use systems offline, this offers a high-capability option previously out of reach on typical developer machines. The Qwen3 base's strong benchmark performance in tool-calling tasks makes the 2-bit tradeoff particularly interesting for this use case compared to smaller base models.