DEV Community

#vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Can vLLM Run GGUF? Yes — on GPU Only
mrsaynothing profile

Can vLLM Run GGUF? Yes — on GPU Only

#ai #llm #gguf #vllm
1
Comments
4 min read
Run vLLM on Kubernetes with Minikube, WSL2 and NVIDIA GPU
vishnuhd profile

Run vLLM on Kubernetes with Minikube, WSL2 and NVIDIA GPU

#ai #localllm #vllm #kubernetes
Comments
13 min read
KV Cache on 16 GB GPUs: Making Long Context Actually Fit
rosgluk profile
Rost

KV Cache on 16 GB GPUs: Making Long Context Actually Fit

#llm #llamacpp #vllm #ollama
Comments 1
22 min read
From adapter to deployment: merging LoRA weights and serving with vLLM or a Space
pranjulrathour profile

From adapter to deployment: merging LoRA weights and serving with vLLM or a Space

#finetuning #deployment #vllm #serving
Comments
3 min read
Inside vLLM: Following One Request from the API to GPU Execution
yuan_lei_e631e36865ed370b profile

Inside vLLM: Following One Request from the API to GPU Execution

#vllm #llm #inference #python
1
Comments 2
24 min read
Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It
xbill profile

Gemma 4 on a Tesla T4, Part 2: The Minimum GCE VM and a Script to Drive It

#gemma #vllm #gcp #cuda
13
Comments
13 min read
vLLM v0.28.0: the breaking change small GPU users must read
induwara_ashinsana_9e4d5b profile

vLLM v0.28.0: the breaking change small GPU users must read

#vllm #llminference #opensource
Comments
5 min read
Serving Gemma 4 on an AMD MI300X: What $1.99 an Hour Buys

Real-world benchmarks and MCP tool management tips
xbill profile

Serving Gemma 4 on an AMD MI300X: What $1.99 an Hour Buys

#amd #vllm #rocm #machinelearning
14
Comments 7
9 min read
Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16
xbill profile

Gemma 4 on a Tesla T4: QAT Weights Decode 1.79x Faster Than bf16

#gemma #vllm #cuda #machinelearning
8
Comments 1
9 min read
FastMCP Is Now MCPServer: Migrating a Python MCP Server to the MCP SDK 2.x

Dependency upgrades break shared environments
xbill profile

FastMCP Is Now MCPServer: Migrating a Python MCP Server to the MCP SDK 2.x

#mcp #python #googlecloud #vllm
14
Comments 10
12 min read
Deploying the 600GB Inkling-NVFP4 Model on Spot A3: A GKE and vLLM Deep Dive
alchemicduncan profile

Deploying the 600GB Inkling-NVFP4 Model on Spot A3: A GKE and vLLM Deep Dive

#gke #vllm #kubernetes #ai
5
Comments
61 min read
Ollama to vLLM: When to Migrate Your Local LLM Server
rosgluk profile
Rost

Ollama to vLLM: When to Migrate Your Local LLM Server

#ollama #vllm #llm #ai
Comments
15 min read
Deploying Inference Using NVIDIA Dynamo and vLLM
sharmukh profile

Deploying Inference Using NVIDIA Dynamo and vLLM

#nvidia #vllm #llm #gpu
13
Comments
8 min read
The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One
xbill profile

The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One

#aws #vllm #cuda #machinelearning
3
Comments
11 min read
DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings
conatusai profile
Jahn

DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings

#nvidia #llm #gpu #vllm
1
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.