A short livestream tutorial on quantization for AI models, introducing concepts and showing practical steps using llama.cpp and vLLM.
Technical data snippet describing per-layer quantization characteristics (Omega, kurtosis, dB) for neural network layers, noting per-channel dynamic activation quantization and standard W8A8 quantization for some layers; sparse bursts dominate the tail across layers.
Quantization reduces numerical precision in LLMs after training, relying on approximations to maintain useful computation. It covers methods like GPTQ, AWQ, and SmoothQuant, each handling weight and activation errors differently to balance accuracy, storage, and execution requirements.
The page explains how 8-bit and 4-bit quantization can reduce image-model file sizes and improve speed, while rounding may alter generated images. It measures how the effects vary by quantization method and model.
ProQuant proposes weight-only mixed-precision post-training quantization for Mixture-of-Experts models, combining pruning with bit-width allocation and routing-consistent finetuning to mitigate quantization errors and routing drift; experiments show improved generation quality with available code.
The document specifies changes to deny -sm tensor usage for arches involved in moe amax rescaling, due to how amax reduction spans matmul rows and the MIRRORED case providing partial per-device max values; it instructs dropping that case and points to TAG_MOE_AMAX_SCALE, with an assist attribution and a hardware context reference.
Release notes describe refactors to conv state handling in delta_net components, moving slot indexing and memory filling to the host, removing device-based ARANGE/argsort mirrors, and crediting a co-author.
PLaMo 3 Translate provides translation-focused LLMs in multiple sizes and quantizations, offering BF16 and various 4- to 8-bit formats for deployment.
Rigetti reported $1.4 million in third-quarter 2026 revenue, a $3.1 million GAAP net loss, and $497.1 million in cash and investments. It also cited 99.5% two-qubit fidelity on its 108-qubit system and a $100 million agreement with the U.S. Department of Commerce.
This repository audit reports incorrect rounding and other errors in IEEE VHDL floating-point and fixed-point packages, using exact arithmetic as a reference. It provides test code, explanations, and patches, and says the float_pkg patch passes more than 32 million calls across four test sets.