Quantization

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Anaz S. Aji for Codecora Dev

Sep 2

A Small Transformer Trained in 1.5 Hours Beat Many LLMs on ARC

#machinelearning #vectorsearch #quantization #benchmark

5 min read

Seth Wheeler

Aug 25

A Better FP4 Gradient Quantizer That Training Couldn't Notice

#llm #measurement #quantization #training

7 min read

Seth Wheeler

Aug 21

Error Feedback, Gradient Compression, and Why Adam Breaks It

#llm #measurement #quantization #training

8 min read

Aomi Qaza

Aug 15

KV Cache INT4 Quantization for 1M+ Token Context Windows

#aiengineering #quantization

3 min read

Seth Wheeler

Aug 22

Comparing INT4 and NVFP4 Palettes on Real Gradient Tensors

#llm #quantization #training #measurement

7 min read

xbill for Google Developer Experts

Aug 19

Deploying a QAT Checkpoint Your Serving Stack Can't Load: Gemma 4 E2B in Pure JAX on One TPU

#tpu #jax #llm #quantization

10 min read

Pneumetron

Jul 17

Unsloth Releases Qwen3.6-27B-NVFP4: Enhanced Throughput and Agentic Coding for Developers

#aiml #largelanguagemodels #quantization #qwen

3 min read

Pneumetron

Jul 15

Bonsai-27B: A 1-Bit LLM for On-Device Inference with Llama.cpp and MLX

#llm #quantization #1bit #gguf

3 min read

JoTeq the First

Jul 14

Quantizing MedGemma to INT4 (GPTQ/W4A16): Everything That Broke Along the Way

#machinelearning #llm #quantization #opensource

6 min read

Aarush Karak

Aug 13

LLM Inference Optimization: From Quantization to Speculative Decoding

#llm #inference #quantization #optimization

2 min read

AI Explore

Jul 7

Self-Hosting a Model Means Self-Hosting Its Evaluation Too

#ai #llm #quantization #mlops

5 min read

byeongsoo kang

Jun 11

Gemma 4 QAT on a 1080 Ti: What 'Quantization-Aware' Actually Buys — and Fitting the 12B on 8 GB at 16k

#llm #machinelearning #gemma #quantization

5 min read

Tech_Nuggets

Jun 11

Quantization formats compared: GGUF vs GPTQ vs AWQ vs NF4

#llm #quantization #mlops #tutorial

7 min read

Kunal

Jul 6

LLM Quantization Levels Compared: Q4_K_M vs Q8_0 vs FP16 [2026]

#localllm #quantization #gguf #ollama

15 min read

Patrick Hughes

Jun 11

How to Pick a GGUF Quant Level for Your VRAM Budget

#localllm #gguf #quantization #gpu

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.