Quantization

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Alan West

May 27

Why your quantized LLM loses its MTP heads and how to keep them

#machinelearning #llm #python #quantization

5 min read

MxGuru

May 20

The Best Result This Week Was a Failed Prediction — Phase-3a Doesn't Transfer

#quantization #hsaq #methodology #granite

1 min read

MxGuru

May 20

Two Localizers, Both Wrong: Bounding a Quantization Cost That Wouldn't Close

#quantization #hsaq #methodology #granite

1 min read

MxGuru

May 20

When the Sensitivity Metric Lies: A Drift-Inversion Smoking Gun in Mixed-Precision LLM Quantization

#quantization #hsaq #awq #granite

8 min read

Patrick Hughes

May 13

GGUF Quantization Explained: Q4_K_M vs Q5_K_M vs Q8 — Which to Pick (2026)

#llamacpp #gguf #quantization #localai

4 min read

Vilius

May 9

1-bit, 545 megabytes, zero API keys — local AI that beats GPT-5.4

#ai #llm #local #quantization

2 min read

Aman Sachan

Apr 30

KVQuant: Run 70B LLMs on 8GB RAM with KV Cache Quantization

#python #llm #quantization

1 min read

Aman Sachan

Apr 30

KVQuant: Run 70B LLMs on 8GB RAM with 4-bit KV Cache Quantization

#python #llm #quantization #optimization

1 min read

Alan West

Apr 18

Traditional Quantization vs 1.58-Bit Ternary Models: A Practical Comparison

#machinelearning #llm #quantization #ai

5 min read

Denis Lavrentyev

Apr 13

GIMP's Posterization: Simple Quantization vs. Median Cut for Better Visuals

#gimp #posterization #quantization #mediancut

8 min read

plasmon

Apr 8

Q4 KV Cache Fit 32K Context into 8GB VRAM — Only Math Broke

#llm #quantization #vram #localllm

8 min read

Jean

May 8

Chasing 16MB: My Parameter Golf Journey and What I Learned the Hard Way

#parametergolf #tinyllm #aiexperimentation #quantization

3 min read

Scott Everitt

Mar 30

Building a Vector Database That Never Decompresses Your Vectors

#vectordatabase #quantization #turboquant #go

16 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# quantization

Why your quantized LLM loses its MTP heads and how to keep them

The Best Result This Week Was a Failed Prediction — Phase-3a Doesn't Transfer

Two Localizers, Both Wrong: Bounding a Quantization Cost That Wouldn't Close

When the Sensitivity Metric Lies: A Drift-Inversion Smoking Gun in Mixed-Precision LLM Quantization

GGUF Quantization Explained: Q4_K_M vs Q5_K_M vs Q8 — Which to Pick (2026)

1-bit, 545 megabytes, zero API keys — local AI that beats GPT-5.4

KVQuant: Run 70B LLMs on 8GB RAM with KV Cache Quantization

KVQuant: Run 70B LLMs on 8GB RAM with 4-bit KV Cache Quantization

Traditional Quantization vs 1.58-Bit Ternary Models: A Practical Comparison

GIMP's Posterization: Simple Quantization vs. Median Cut for Better Visuals

Q4 KV Cache Fit 32K Context into 8GB VRAM — Only Math Broke

Chasing 16MB: My Parameter Golf Journey and What I Learned the Hard Way

Building a Vector Database That Never Decompresses Your Vectors