Underdog compresses Qwen3.8-27B to 7.89GB: self-tested tool calling surpasses the original version

动察 Beating AI News Flash: a16z-backed local AI assistant Underdog releases Saluki 27B, built on Alibaba's Qwen3.8-27B. The model uses approximately 2-bit quantization, reducing its size from about 54GB for the full-precision version to 7.89GB. The company says it can run on a laptop with 16GB of memory and supports deployment through existing tools such as llama.cpp. The model weights have been released under the Apache 2.0 license.

Saluki uses quantization results from ISTA-DASLab at the Institute of Science and Technology Austria. The lab had previously compressed Qwen3.8-27B to about 8.4GB. Underdog further reduced the size on this basis and focused on optimizing tool-calling performance.

Among 120 tool-calling questions selected from BFCL, Saluki answered 88 correctly, the ISTA quantized version answered 76 correctly, and the 54GB original version answered 84 correctly. In 50 real GitHub issue-fixing tasks, Saluki solved 30, while the original version solved 33.

The company says Saluki retained an average of 96% of the original version's performance across 9 evaluations, but performance varied widely across different tasks. Math competition and complex reasoning scores reached only about 83% and 85% of the original version, respectively.