According to Dynamic Beating monitoring, the open-source AI tool company Unsloth has released the 1-bit GGUF version of Kimi K3, compressing the model from 1.56TB to 594GB. Users can run it locally on a Mac through Unsloth Studio or llama.cpp.
However, this version requires at least about 610GB of memory. Following the official recommendation, the top-of-the-line 512GB Mac Studio needs to be paired with a 128GB memory device. Insufficient memory can also be supplemented by loading from a hard drive, but this will result in a significant speed reduction.
The community also offers 350GB and 451GB MLX trimmed versions that can fit into a single 512GB Mac Studio. However, they have removed 73% to 80% of the experts, with observed speeds of approximately 0.14 to 0.20 tokens per second, showing a noticeable decrease in performance.
