Back to all stories

The Llama-modchip project has achieved a breakthrough by running large AI models, including the 284B DeepSeek-V4-Flash and 30B MoEs, on a standard 64GB CPU without the need for a GPU. This is made possible by optimizing the models to run within the 64GB RAM limit, achieving speeds of up to 44.4 tokens per second for DeepSeek-V4-Flash and 111 tokens per second for MoEs. The project provides a range of tools and resources, including a quickstart guide, supported models, and a leaderboard, making it possible for users to run these models on their existing CPU workstations without requiring any specialized hardware or NVMe swap.