AMD plans to integrate Taalas’s silicon-embedded AI model chips into its infrastructure, aiming to overcome memory bottlenecks and revolutionise AI inference performance with specialised hardware solutions.
AMD has agreed to buy Taalas, a Toronto-based startup whose chips embed an AI model’s weights directly into silicon, a design that aims to sidestep the memory bottleneck that limits many GPU-based inference systems. The deal, announced on August 6, is due to close in the fourth quarter of 2026 if regulators approve it. AMD said the purchase fits its broader push to offer a more complete AI stack, and its shares rose about 1.5% after the news.
Taalas is trying to solve a basic problem in large language model deployment: the cost of moving data is often more important than the cost of doing the maths. In normal inference, a model must repeatedly read its weights from high-bandwidth memory for each generated token. Taalas’s approach, which it calls Hard Coded Inference, makes those weights part of the chip itself. The company’s HC1 demonstrator hardwires Meta’s Llama 3.1 8B model into a large die built on TSMC’s 6nm process. In February, Taalas said HC1 could reach about 17,000 tokens per second per user, but those figures have not been independently verified at production scale.
That performance comes with a sharp trade-off. A chip that contains one model in hardware cannot readily run another. Taalas has argued that its structured-ASIC method could cut the turnaround for a new model to roughly two months by changing only a small number of fabrication layers. Even so, the company’s first chip uses a proprietary 3-bit quantisation scheme that can reduce output quality on more demanding tasks, according to technical reviews. A second-generation chip, HC2, is expected between winter 2026 and early 2027 and is meant to support a more standard 4-bit format and larger models.
AMD plans to fold Taalas into its Helios rack-scale systems alongside Instinct GPUs and EPYC CPUs, using the ROCm software stack. The idea is to split inference into two stages: GPUs would handle prefill, when a prompt is processed, while Taalas chips would handle decode, when tokens are generated one by one and memory bandwidth becomes the main constraint. The strategy reflects a wider shift in the AI hardware market, as vendors increasingly move beyond general-purpose GPUs towards specialised inference silicon. It also leaves AMD with a promising technical direction, but not yet a proven production product at scale.
Disclaimer: This content is intended for informational purposes only. Readers are advised to exercise their own judgement, conduct due diligence, or consult a qualified expert before acting on any information provided.
