DEV Community

#cuda

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Comparing Qwen and Gemma on the Low End: Qwen Is Faster Without Drafting, Gemma Has Better Quality
xbill profile

Comparing Qwen and Gemma on the Low End: Qwen Is Faster Without Drafting, Gemma Has Better Quality

#gemma #qwen #llamacpp #cuda
Comments
13 min read
Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050
xbill profile

Can a QAT 12B Fit an 8 GB GPU? Gemma 4 12B on an RTX 3050

#gemma #llamacpp #cuda #benchmarking
1
Comments
15 min read
Running NVIDIA's new 120B Nemotron-3.5-Super on an 8 GB GPU — skipping the zeros, and finding the real bottleneck
helgard_orlm profile

Running NVIDIA's new 120B Nemotron-3.5-Super on an 8 GB GPU — skipping the zeros, and finding the real bottleneck

#ai #llm #cuda #python
Comments
7 min read
I read a CUDA inference engine's symbols with Ghidra, then made it 12% faster on my 3090
prodbymarcu profile

I read a CUDA inference engine's symbols with Ghidra, then made it 12% faster on my 3090

#python #cuda #llm #inference
Comments
5 min read
I wrote 21 CUDA kernels on a free GPU. Here's what the profiler taught me.
musharaf_khanpathan_90de profile

I wrote 21 CUDA kernels on a free GPU. Here's what the profiler taught me.

#cuda #gpu #performance #cpp
Comments
5 min read
Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

Acceptance rates hold steady across hardware
xbill profile

Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

#gemma #llamacpp #cuda #benchmarking
8
Comments 3
19 min read
Running vLLM natively on Windows
aivrar profile
aivrar

Running vLLM natively on Windows

#vllm #windows #cuda #llm
Comments
6 min read
Running the 510 GB DeepSeek-V4.1-Flash on an 8 GB GPU — and three bugs that never raise an error
helgard_orlm profile

Running the 510 GB DeepSeek-V4.1-Flash on an 8 GB GPU — and three bugs that never raise an error

#ai #llm #cuda #python
Comments
8 min read
Running a 133 GB MoE model on an 8 GB GPU at 11 tokens/s by streaming experts from NVMe
helgard_orlm profile

Running a 133 GB MoE model on an 8 GB GPU at 11 tokens/s by streaming experts from NVMe

#ai #llm #cuda #python
Comments
6 min read
Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

Uses a 2014 desktop and custom MCP tools
xbill profile

Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

#gemma #llamacpp #cuda #localllm
11
Comments 6
13 min read
How Fast Can a 421M-Parameter Decision Model Run? I Benchmarked Laya Across NVIDIA GPUs
cookies_c9dc8b91f33d29250 profile

How Fast Can a 421M-Parameter Decision Model Run? I Benchmarked Laya Across NVIDIA GPUs

#machinelearning #cuda #performance #opensource
Comments 1
6 min read
Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes
hrlpavan profile

Zero-Cloud AI Code Review: Connecting Self-Hosted GitLab to Local CUDA LLM Runtimes

#gitlab #ai #cuda #selfhosted
Comments
4 min read
Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture
comradepenguin1917 profile

Exploring Result Visibility of Fixed-Latency Instructions on the SM120 Architecture

#cuda #nvidia #gpu
1
Comments
10 min read
Nvidia Just Let Rust Into CUDA. Here's Why That's a Bigger Deal Than It Sounds
ashraf_chowdury09 profile

Nvidia Just Let Rust Into CUDA. Here's Why That's a Bigger Deal Than It Sounds

#rust #ai #cuda #programming
1
Comments
5 min read
CUDA Rust: two native tracks, not a wrapper over C++
jtorchia profile

CUDA Rust: two native tracks, not a wrapper over C++

#english #rust #cuda #gpuprogramming
1
Comments
5 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.