Benchmarks - DEV Community

Skip to content

DEV Community

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Peremptory

Sep 1

DeepSeek's Vision Flash Plays the Agent Game

#modelrelease #chineseai #agenticai #benchmarks

2 min read

Peremptory

Aug 25

Frontier Models Fail at Research-Level Reasoning

#benchmarks #research #aidevelopment

2 min read

Sep 5

Temperature 0 is not reproducible. I measured 30 percent of my output changing between identical runs.

#llm #benchmarks #localllm #reproducibility

3 min read

Sep 5

Our 4B beat Claude Opus on a 440K-token corpus. Then it came last on the public benchmark.

#llm #benchmarks #localllm #privateai

4 min read

Sep 5

A 4B on a 6GB laptop matched frontier-model accuracy on aggregation — except when the answer is a number

#llm #benchmarks #localllm #agents

4 min read

Sep 5

Your agent truncates the corpus and answers anyway. Two harnesses, and a router that picks between them.

#llm #agents #localllm #benchmarks

5 min read

Peremptory

Aug 21

Frontier Models Hit a Wall on Research Thinking

#research #benchmarks #aidevelopment

2 min read

Aug 18

Ranking Language Models by How Well They Spot Liars

#llm #measurement #benchmarks #statistics

9 min read

Charles

Aug 18

The Benchmarkpocalypse: Why AI Benchmarks Are Broken — and What Dan Luu Says We Should Do About It

#ai #machinelearning #benchmarks #programming

4 min read

Peremptory

Aug 18

Labs are ditching factual knowledge for reasoning speed

#benchmarks #modelreleases #aistrategy #research

2 min read

The AI Downside

Aug 15

Why AI Benchmarks Mean Less Than You Think

#benchmarks #llms #evaluation #hype

6 min read

Aug 15

An AI Capture-the-Flag Tournament: What the Scoreboard Counted

#llm #measurement #security #benchmarks

6 min read

Tommy Leonhardsen

Aug 11

Twelve LLMs Played Werewolf. The Real Wolf Was the Thinking Knob.

#llm #benchmarks #reasoning #latency

8 min read

Peremptory

Aug 7

Meta's Muse Code Clears 59% on Deep Software Engineering

#modelrelease #developertools #benchmarks #codingmodels

2 min read

Edward Izgorodin

Aug 29

The Same GraphRAG Comparison Wins and Loses. It Depends Which Instrument Judged It.

#ai #rag #llm #benchmarks

3 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.