Kimi K2.5 Runs on RTX 3060 with 768GB Intel Optane Memory at 4 Tokens per Second

1-Trillion-Parameter LLM on a Single GPU with 768GB Cheap Intel Optane DIMMs: Kimi K2.5 at ~4 tokens/sec

1-Trillion-Parameter LLM on a Single GPU with 768GB Intel Optane DIMMs – “Kimi K2.5 at ~4 tokens/sec” Explained 1-Trillion-Parameter LLM on a Single GPU with 768GB Cheap Intel Optane DIMMs: “Kimi K2.5 at ~4 tokens/sec” Editor’s note: This article unpacks the engineering behind serving a trillion-parameter-class LLM on a single GPU by leveraging a large pool of Intel Optane persistent memory (PMem). We treat “Kimi K2.5 at ~4 tokens/sec” as a case…

Kimi K2.5 runs on RTX 3060 with 768GB Intel Optane memory at 4 tokens per second

This experiment highlights the potential for democratizing AI access, enabling advanced models to run on more affordable, widely available hardware. The post Kimi K2.5 runs on RTX 3060 with 768GB Intel Optane memory at 4 tokens per second appeared first on Crypto Briefing.

768GB of cheap Intel Optane DIMM memory sticks used to run 1-trillion-parameter LLM on a system with a single GPU — local Kimi K2.5 install achieved roughly 4 tokens per second

A Redditor has caused a stir by coaxing a workstation build using Optane PMem DIMMs as RAM to run a 1-trillion parameter LLM.