DEV Community

InferenceKey profile picture

InferenceKey

Open models on your own GPUs: experiments, measurements and the exact commands behind them. Full write-ups at inferencekey.com

Joined Joined on  Personal website https://inferencekey.com
Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned
inferencekeydev profile

Serving a 125B coding model to a team from one RTX 5090: what broke and what we tuned

#ai #llm #gpu #opensource
1
Comments
3 min read

Want to connect with InferenceKey?

Create an account to connect with InferenceKey. You can also sign in below to proceed if you already have an account.

Create Account
Already have an account? Sign in
Kandinsky 6.0 in a pipeline: offload, the untimed default and what to measure
inferencekeydev profile

Kandinsky 6.0 in a pipeline: offload, the untimed default and what to measure

#ai #machinelearning #opensource #gpu
Comments
4 min read
A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream
inferencekeydev profile

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

#llm #gpu #machinelearning #selfhosted
Comments 2
4 min read
Same config.json, different serve command: what to check before serving AREX-2
inferencekeydev profile

Same config.json, different serve command: what to check before serving AREX-2

#ai #llm #machinelearning #gpu
Comments
5 min read
Three things that break when you point Claude Code at your own model
inferencekeydev profile

Three things that break when you point Claude Code at your own model

#ai #llm #selfhosted #opensource
Comments 2
4 min read
loading...