DEV Community

#vllm

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
10
Comments
9 min read
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
Comments
9 min read
Ollama to vLLM: When to Migrate Your Local LLM Server
rosgluk profile
Rost

Ollama to vLLM: When to Migrate Your Local LLM Server

#ollama #vllm #llm #ai
Comments
15 min read
The unofficial TPU migration guide: Cloud TPU API to Compute Engine
xbill profile

The unofficial TPU migration guide: Cloud TPU API to Compute Engine

#tpu #gcp #vllm #devops
6
Comments 2
17 min read
Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity
xbill profile

Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity

#tpu #llm #vllm #antigravity
Comments
8 min read
Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1
xbill profile

Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1

#tpu #vllm #llm #gcp
15
Comments 1
21 min read
What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage
mingxintech profile

What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage

#kvcache #lmcache #vllm #ai
Comments
5 min read
Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't
xbill profile

Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't

#tpu #vllm #llm #gcp
2
Comments
20 min read
Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)
sikamikanikobg profile

Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)

#llm #ollama #vllm #gpu
Comments
3 min read
vLLM vs llama.cpp vs Ollama: What Happens When Your Model Doesn't Fit in 24GB VRAM
sikamikanikobg profile

vLLM vs llama.cpp vs Ollama: What Happens When Your Model Doesn't Fit in 24GB VRAM

#llm #homelab #vllm #ai
Comments
6 min read
Gemma 4 E2B on a Single TPU v6e Chip: A Serving Deep Dive
xbill profile

Gemma 4 E2B on a Single TPU v6e Chip: A Serving Deep Dive

#tpu #llm #vllm #googlecloud
8
Comments 1
8 min read
tpu-management: a Claude Code skill for running Gemma 4 on Cloud TPUs
xbill profile

tpu-management: a Claude Code skill for running Gemma 4 on Cloud TPUs

#googlecloud #tpu #vllm #claudecode
7
Comments
3 min read
AI Inference Optimization in Cloud-Native Environments: GPU Orchestration, Edge Deployment, and Latency Reduction at Scale
thecybersidekick profile

AI Inference Optimization in Cloud-Native Environments: GPU Orchestration, Edge Deployment, and Latency Reduction at Scale

#kubernetesgpuscheduling #aiinferenceoptimization #llmdeployment #vllm
Comments
4 min read
Kimi K3 Open Weights Are Here: How to Self-Host the 2.8T-Parameter Model (Hardware, vLLM, and Data Sovereignty)
lola_lin_a1be8395c517b081 profile

Kimi K3 Open Weights Are Here: How to Self-Host the 2.8T-Parameter Model (Hardware, vLLM, and Data Sovereignty)

#kimik3 #openweights #selfhostai #vllm
4
Comments
8 min read
AI Inference at the Edge: Running Real-Time LLMs in Kubernetes Without a GPU Farm
thecybersidekick profile

AI Inference at the Edge: Running Real-Time LLMs in Kubernetes Without a GPU Farm

#edgeai #kubernetes #llminference #vllm
Comments
3 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.