DEV Community

#localllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090
seppegadeyne profile

Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090

#ai #localllm #llm #tutorial
Comments
10 min read
Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM
seppegadeyne profile

Run Qwen3.8-Flash-Next at 79 tok/s with Strata and a 5-bit quant on 128 GB of RAM

#ai #localllm #llm #tutorial
Comments
11 min read
Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s
jamilxt profile

Run a 48 GB AI Model on a 32 GB Laptop: Janas Streams Model Experts From Your SSD at 37 tok/s

#ai #localllm #performance #llm
1
Comments
7 min read
Orquesta CLI: Local LLM Management with Dashboard Sync
orquesta_live profile

Orquesta CLI: Local LLM Management with Dashboard Sync

#orquestacli #localllm #dashboardsync #ollama
Comments
6 min read
Same Local Model, 3 Coding Agents: pi 12/12, Codex 12/12, Claude Code 7/12 on a 6GB GPU
kunfieldnotes profile

Same Local Model, 3 Coding Agents: pi 12/12, Codex 12/12, Claude Code 7/12 on a 6GB GPU

#localllm #llamacpp #ai #claudecode
2
Comments 4
3 min read
What broke when we ran agents on small local models
beesbot profile
Bees

What broke when we ran agents on small local models

#localllm #ai #agents #opensource
1
Comments 2
6 min read
Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

Uses a 2014 desktop and custom MCP tools
xbill profile

Gemma 4 on a Used 8 GB RTX 3050: 116 Tokens/s for One User, 618 for 64

#gemma #llamacpp #cuda #localllm
10
Comments 6
13 min read
Estimating tokens/s for Mixture-of-Experts models: active parameters, plus a routing term
alexfank profile

Estimating tokens/s for Mixture-of-Experts models: active parameters, plus a routing term

#llm #localllm #machinelearning #performance
1
Comments
5 min read
EmbeddingGemma 2 puts multimodal search on a phone
techaiwire profile

EmbeddingGemma 2 puts multimodal search on a phone

#google #openweights #localllm #huggingface
5
Comments
4 min read
Qwen3.8-27B on one RTX 3090, and the setting its README says chat clients should leave at default
renolu profile

Qwen3.8-27B on one RTX 3090, and the setting its README says chat clients should leave at default

#vllm #localllm #speculativedecoding #quantization
Comments
4 min read
Still carrying the diagnosis
thekilteddev profile

Still carrying the diagnosis

#localllm #debugging #buildinpublic #opensource
Comments
8 min read
Claude CLI 401 Unauthorized Refresh Token Issue
devlog profile

Claude CLI 401 Unauthorized Refresh Token Issue

#clitools #localllm #troubleshooting
Comments
3 min read
Discord bot not responding but still running: how to catch it
devlog profile

Discord bot not responding but still running: how to catch it

#automation #localllm #macos
Comments
3 min read
Claude Code MCP setup and usage rules
devlog profile

Claude Code MCP setup and usage rules

#clitools #macos #localllm
Comments
3 min read
Clone your own voice with a 17-second recording
devlog profile

Clone your own voice with a 17-second recording

#contentops #localllm #macos
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.