Musk’s AI company xAI officially releases the Grok-1 source code

According to BlockBeats news, on March 18, Musk’s artificial intelligence company xAI announced the opening of the weights and architecture of its 314 billion parameter hybrid expert model Grok-1. Grok-1 is a large language model trained from scratch by xAI, using a hybrid expert model architecture where 25% of the weights are active on a given token. The model's basic model weights and network architecture are now available under the Apache 2.0 license. It is reported that the pre-training phase of Grok-1 ended in October 2023, and what is currently released is the original basic model checkpoint, which has not been fine-tuned for any specific application.