DEV Community

# vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Running LLMs on Windows: Native vLLM vs WSL vs llama.cpp Compared
alanwest profile

Running LLMs on Windows: Native vLLM vs WSL vs llama.cpp Compared

#llm #vllm #windows #machinelearning
Comments
4 min read
The 70B Threshold: How the RTX 5090 Rewrites the Home Lab Equation
glad_labs profile

The 70B Threshold: How the RTX 5090 Rewrites the Home Lab Equation

#model #memory #models #vllm
Comments
8 min read
11-Second Time to First Token on a Healthy vLLM Server
ingero profile

11-Second Time to First Token on a Healthy vLLM Server

#vllm #observability #ebpf #mcp
1
Comments
5 min read
Self-hosted Gemma 4 on TPU with vLLM, MCP, ADK, and Gemini CLI
xbill profile

Self-hosted Gemma 4 on TPU with vLLM, MCP, ADK, and Gemini CLI

#vllm #googleadk #tpu #gemini
26
Comments
16 min read
Performance Test: Ollama 0.5.0 vs. vLLM 0.4.0 Local LLM Inference Latency on NVIDIA RTX 5090 and AMD Radeon RX 8900 in 2026
johalputt profile

Performance Test: Ollama 0.5.0 vs. vLLM 0.4.0 Local LLM Inference Latency on NVIDIA RTX 5090 and AMD Radeon RX 8900 in 2026

#performance #test #ollama #vllm
Comments
14 min read
Why We Stopped Using vLLM 0.6 for Local LLMs in Favor of Ollama 0.5 for Code Tasks
johalputt profile

Why We Stopped Using vLLM 0.6 for Local LLMs in Favor of Ollama 0.5 for Code Tasks

#stopped #using #vllm #local
Comments
14 min read
Comparison: vLLM 0.6 vs. Text Generation Inference 1.4 for Serving Code LLMs
johalputt profile

Comparison: vLLM 0.6 vs. Text Generation Inference 1.4 for Serving Code LLMs

#comparison #vllm #text #generation
Comments
16 min read
From one model to seven — what it took to make TurboQuant model-portable
albertocodes profile

From one model to seven — what it took to make TurboQuant model-portable

#python #vllm #gpu #triton
Comments
3 min read
Compressed VLM inference from a single Containerfile — turboquant-vllm v1.1
albertocodes profile

Compressed VLM inference from a single Containerfile — turboquant-vllm v1.1

#python #vllm #gpu #containers
1
Comments
2 min read
vLLM Request Lifecycle (Where TTFT is measured)
pilgrim2go profile

vLLM Request Lifecycle (Where TTFT is measured)

#vllm #monitoring
Comments
2 min read
How to Run Gemma 4 Locally With Ollama, llama.cpp, and vLLM
dmaxdev profile

How to Run Gemma 4 Locally With Ollama, llama.cpp, and vLLM

#gemma4 #ollama #llamacpp #vllm
2
Comments 1
9 min read
Gemma-SRE: Self-Hosted vLLM Infrastructure Agent
xbill profile

Gemma-SRE: Self-Hosted vLLM Infrastructure Agent

#gemma #mcpserver #tpusprint #vllm
1
Comments
18 min read
vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs
soytuber profile
soy

vLLM On-Demand Gateway: Zero-VRAM Standby for Local LLMs on Consumer GPUs

#vllm #llm #gpu #python
2
Comments 1
4 min read
I Pushed Local LLMs Harder. Here's What Two Models Actually Did.
dcruver profile

I Pushed Local LLMs Harder. Here's What Two Models Actually Did.

#claudecode #vllm #selfhosted #amd
1
Comments
8 min read
Pare de Brincar com LLMs Locais: Leve a IAG Open Source para a Produção na Magalu Cloud
glaucioripol profile

Pare de Brincar com LLMs Locais: Leve a IAG Open Source para a Produção na Magalu Cloud

#ai #llm #vllm #docker
2
Comments 3
22 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.