Beating AI News Flash: AMD announced the completion of its acquisition of Canadian AI chip company Taalas. The two parties reached an agreement in August this year, with the total transaction value of up to approximately $2.9 billion, of which up to $1.4 billion will be paid only upon meeting agreed conditions.
Taalas has taken a completely different route from traditional GPUs: when manufacturing the chip, the parameters of the AI model are directly solidified into the circuitry.
Ordinary GPUs need to repeatedly read model parameters from video memory. Taalas puts storage and computing together, reducing data movement and also eliminating the need for expensive high-bandwidth memory (HBM) and related complex packaging.
In February this year, Taalas launched its first chip, HC1, specifically designed to run Meta's Llama 3.1 8B model. According to official tests, under the condition of 1,000 tokens each for input and output, the single-user generation speed reached about 17,000 tokens per second, nearly 9 times that of Cerebras at the time.
But this design also sacrifices flexibility. Ordinary GPUs can load different models at any time. Although HC1 supports LoRA fine-tuning, changing the base model still requires modifying the manufacturing photomask and re-producing the chip. The cycle given by Taalas is about two months.
HC1 also adopts aggressive 3-bit and 6-bit quantization. Although it greatly compresses model parameters, it also loses some model quality. At present, the result of 17,000 tokens per second comes only from Llama 3.1 8B, and the actual performance of larger models remains to be verified.
AMD plans to incorporate this technology into future AI accelerator products, to be used in conjunction with its existing Instinct GPUs, allowing specialized chips to handle high-frequency inference tasks for fixed models.
