New KV cache compaction technique cuts LLM memory 50x without accuracy loss

MIT researchers developed Attention Matching, a KV cache compaction technique that compresses LLM memory by 50x in seconds — ...

Nvidia says it can shrink LLM memory 20x without changing model weights

Nvidia's KV Cache Transform Coding (KVTC) compresses LLM key-value cache by 20x without model changes, cutting GPU memory ...
TMCnet

Breaking the 100M Token Limit: EverMind's MSA Architecture Achieves Efficient End-to-End Long-Term Memory for LLMs

This approach can be viewed as a memory plug-in for large models, providing a fresh perspective and direction for solving the ...
TechSpot

AMD 3D V-Cache CPU memory used to create incredibly fast RAM disk

Why it matters: A RAM drive is traditionally conceived as a block of volatile memory "formatted" to be used as a secondary storage disk drive. RAM disks are extremely fast compared to HDDs or even ...
Electronic Design

Hardware Compression Works at the Memory Cache Level

How lossless data compression can reduce memory and power requirements. How ZeroPoint’s compression technology differs from the competition. One can never have enough memory, and one way to get more ...
TMCnet

Penguin Solutions Introduces Industry's First Production-Ready CXL-Based KV Cache Server

Accelerating memory-dependent AI processes, Penguin's MemoryAI KV cache server increases memory capacity by integrating 3 TB ...

New memory architecture targets AI inference bottlenecks

Lightbits Labs Ltd. today is introducing a new architecture aimed at addressing one of the most stubborn bottlenecks in large ...
FullCleared on MSN

AMD's 9800X3D processor anchors this high-memory gaming build

The latest Area-51 desktop from Alienware centers around AMD’s Ryzen 7 9800X3D, an 8-core processor with 104MB of total cache ...