Search

AI Papers Daily: research papers worth building on

LowRAM compresses KV caches by predicting values from keys and using low-rank key compression. The paper reports up to 5× faster decoding at 4× compression.

[2610.27913] ACE-Compression: An Ability-Centric Evaluation of Compressed Large Language Models

ACE-Compression assesses how compression changes the core abilities of large language models, beyond overall accuracy. It shows that quantization often preserves broader abilities better than pruning, and recommends choosing compression methods based on the specific abilities required by an application, turning compression into a measurable, interpretable process.

[2611.07025] LowRAM: Low-Rank Keys and Amortised Values for Composable KV Cache Compression at Test-Time

LowRAM compresses keys and values with separate methods: low-rank key compression and test-time prediction of values from keys. It reports up to fivefold higher peak decode throughput at fourfold compression, and 26.9-fold compression at 94.5% of baseline performance when combined with quantisation and eviction.

[2611.06380] Does Context Compression Preserve Refusal Alignment?

Context compression lowers inference cost but can degrade refusal alignment in instruction-tuned models; a decoder-refusal signal is weakened in compressed space, and a memory-steering approach can recover refusal behavior without retraining, implying semantic fidelity alone is insufficient for preserving alignment.

[2610.31574] Rethinking Sequence Compression with Test-time Regression

The paper presents TTR-Compressor, a test-time regression approach for sequence compression that treats input tokens as a regression problem and computes coefficients in closed form for efficiency. It outperforms traditional pooling/ projection methods in block-wise sparse attention and visual token merging, reducing token counts substantially while maintaining performance.

[2611.06858] CompressionBench: How can your coding agents compress LLMs?

The paper studies autonomous coding agents compressing pretrained models within storage and time constraints, achieving competitive perplexity and downstream accuracy across multiple tasks and settings, while highlighting design, judgment, and validation as central to effective model compression.

DeepSeek Harness compaction: early trigger and stuck switch

DSH context compaction fails and slows due to a fixed headroom and a threshold that can trigger too early; switching to a smaller window causes session-wide 400s errors because summaries exceed the new model's limits. Authors propose per-model configuration, lower headroom, smaller summary tokens, and optional pruning to fix.

Why does Japanese consume tokens? Counting the mechanism by which kanji are split into bytes with tiktoken

Translated title. Original: 日本語はなぜトークンを食う？ 漢字がバイトに割れる仕組みを tiktoken で数えてみた

The document explains that AI models count tokens, not characters, affecting cost and context length. It shows that Japanese text often yields more tokens than English due to UTF-8 bytes and tokenizer behavior, but newer models narrow this gap; it provides methods to count tokens and notes tokenizer differences across models.

[2611.12426] dNaViT: Discrete Codes are General-Purpose Visual Representations

The paper presents dNaViT, a visual tokenizer that represents images as discrete codes while retaining semantic and pixel-level information. It reports results matching or exceeding task-specific baselines across visual understanding, generation, and image editing benchmarks.

Tokens and context window, how the LLM really works

Translated title. Original: Tokeny i okno kontekstowe, jak naprawdę działa LLM

The document describes how LLMs rely on tokens and context windows, detailing tokenization, costs, and how language and formatting affect token counts. It explains that models predict the next token, with tokenizers (including OpenAI and Hugging Face variants) converting text into token sequences, and that BPE underlies most tokenizers.