DEV Community

#vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Deploying Inference Using NVIDIA Dynamo and vLLM
sharmukh profile

Deploying Inference Using NVIDIA Dynamo and vLLM

#nvidia #vllm #llm #gpu
6
Comments
8 min read
vLLM v0.28.0: the breaking change small GPU users must read
induwara_ashinsana_9e4d5b profile

vLLM v0.28.0: the breaking change small GPU users must read

#vllm #llminference #opensource
Comments
5 min read
Installing Rust for vLLM on Graviton: a G5g walk-through 🦀
xbill profile

Installing Rust for vLLM on Graviton: a G5g walk-through 🦀

#rust #vllm #aws #cuda
Comments
10 min read
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
Comments
9 min read
The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One
xbill profile

The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One

#aws #vllm #cuda #machinelearning
2
Comments
11 min read
DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings
conatusai profile
Jahn

DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings

#nvidia #llm #gpu #vllm
Comments
2 min read
Ollama to vLLM: When to Migrate Your Local LLM Server
rosgluk profile
Rost

Ollama to vLLM: When to Migrate Your Local LLM Server

#ollama #vllm #llm #ai
Comments
15 min read
The unofficial TPU migration guide: Cloud TPU API to Compute Engine
xbill profile

The unofficial TPU migration guide: Cloud TPU API to Compute Engine

#tpu #gcp #vllm #devops
6
Comments 2
17 min read
Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity
xbill profile

Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity

#tpu #llm #vllm #antigravity
Comments
8 min read
What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage
mingxintech profile

What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage

#kvcache #lmcache #vllm #ai
Comments
5 min read
Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't
xbill profile

Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't

#tpu #vllm #llm #gcp
2
Comments
20 min read
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
13
Comments
9 min read
Serving Gemma4 with Rust on vLLM 🦀
xbill profile

Serving Gemma4 with Rust on vLLM 🦀

#rust #vllm #aws #cuda
10
Comments
10 min read
Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1
xbill profile

Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1

#tpu #vllm #llm #gcp
16
Comments 1
21 min read
Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)
sikamikanikobg profile

Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)

#llm #ollama #vllm #gpu
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.