cache-eviction

Easy control for Key-Value Constrained Generative LLM Inference(https://arxiv.org/abs/2402.06262)

SIEVE cache - lockfree, concurrent cache - faster and simpler than LRU

A comprehensive open-source cache trace dataset

Hardware-level security protocol for mitigating cache-based side-channel attacks.

A prefix-cache advisor for LLM serving infrastructure that recommends KV-cache capacity and eviction policies from your request traces/logs.

A learned, cost-aware eviction policy for semantic LLM caches that replaces heuristic policies like LRU/FIFO with a lightweight offline-trained model, improving regeneration-token savings while maintaining safe LRU fallback.

Trace-driven research harness for KV-cache hierarchy policy evaluation in long-context LLM inference.

Idle but not forgotten: usage-driven agent memory forgets the dormant skills you'll need next quarter; NicheMem's competitive ownership retains them exactly. Mechanism-level benchmark + verified theory (sibling of GAUSE).

CLOCK replacement algorithm, an efficient, compact and concurrent alternative to LRU cache eviction policy.

REAL: REtrieval-reAsoning and Logic-constructed Attention Behaviors for Long-Context KV Cache Compression

An implementation of W-TinyLFU policy

HybridCacheSystem is a modern C++ framework for building high-performance, multi-layered cache systems. It supports both LRU and LFU caching strategies, allows combining them for hybrid setups, and provides a unified interface for cache management and customization.

Event-driven simulator for prefix KV-cache eviction policies in LLM serving systems

Trace-driven simulator for learned KV Cache eviction on synthetic sparse block-level access traces

Educational caching reverse proxy in Go—TTL-based in-memory HTTP cache, size-aware eviction, gzip negotiation, Cache-Control handling, hit/miss headers, and JSON stats.

LAFC-Evict: counterfactual supervision dataset and benchmark for learned cache eviction

A small caching library implementing standard caching strategies, starting with LRU, in Python. Part of the multi-language mnemosyne-cache project. Built by hand to develop real, working fluency in caching mechanics rather than just theory.

LRU cache in Rust with TTL expiry, a capacity bound, and live hit-rate stats, O(1) get and put

A small caching library implementing standard caching strategies, starting with LRU, in Go. Part of the multi-language mnemosyne-cache project. Built by hand to develop real, working fluency in caching mechanics rather than just theory.

Neutral, reproducible comparison of Go in-process cache libraries: hit rate and per-op cost across synthetic workloads and real traces, with the provenance to regenerate every number.