According to Dynamic Watch Beating monitoring, Google has released a quantized compression version of the model weights for the Gemma 4 large model. Quantization is a technique that reduces model size by lowering numerical precision, but it often leads to a decrease in model performance. To address the issue of precision loss, Google has employed quantization-aware training in this release, where the compression state is simulated early in the training process, allowing the model to automatically adjust to operate at lower precision. This approach enables the model to significantly reduce its size while maintaining a high level of intelligence.
Through specialized mobile compression technology, the Gemma 4 E2B pure text model's memory footprint for running on mobile and edge devices can be reduced to below 1GB. To ensure smooth local execution on mobile phones, Google has specifically optimized the mobile chips. The development team offloads most of the computation during training to alleviate the real-time burden on the mobile chips, and utilizes a computation structure that is more suitable for mobile hardware for native acceleration. The team has also deeply compressed the vocabulary and cache. Furthermore, extreme 2-bit compression has been implemented for layers responsible for generating Tokens, retaining high precision only in the core inference layer. This maximizes storage savings without compromising core logic performance.
The new model weights are now open-source on Hugging Face. Individual users can directly download and run them on their personal computers through Ollama and LM Studio. For mobile and web developers, the new weights are synchronized to support the LiteRT-LM engine, and they can be run directly on the web through Transformers.js. Additionally, vLLM, SGLang, and the Apple chip acceleration tool MLX have been made compatible for quick developer deployment.
