Evaluation

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Maya Andersson

Jul 8

One average eval score was hiding two different failure modes

#evaluation #statistics #analytics #datascience

5 min read

Puneet Gupta

Jul 5

Evaluating LLM Apps in Java

#java #ai #llm #evaluation

10 min read

Puneet Gupta

Jul 5

Evaluating LLM Apps in Python

#python #ai #llm #evaluation

9 min read

AI Explore

Jul 5

The Mean Is Lying to You: Benchmarks Hide the Variance That Breaks Prod

#ai #llm #evaluation #benchmarking

5 min read

Maya Andersson

Jul 7

Your LLM-as-judge has a position bias you are not measuring

#llm #evaluation #statistics #machinelearning

6 min read

Saurav Bhattacharya

Jul 2

Short-Circuit Your Agent Evals: Tier Order Is a Latency Budget, Not a Preference

#ai #agents #evaluation #typescript

5 min read

Breach Protocol

Jul 1

Your AI judge might be reliable — and still be wrong

#evaluation #llmjudges #rlhf #methodology

3 min read

Breach Protocol

Jul 1

Reliable, and still wrong

#evaluation #llmasjudge #benchmarks

3 min read

Saurav Bhattacharya

Jun 29

Give Your Agent a Type Signature: Contract-First Output Beats a Smarter Judge

#ai #agents #evaluation #typescript

4 min read

Saurav Bhattacharya

Jun 28

Your Model-as-Judge Doesn't Belong in the Hot Path

#ai #agents #evaluation #observability

9 min read

Tanishq Soni

Jun 28

Evaluating Large Language Models: The Pitfall of Overfitting in RAG

#llm #evaluation #overfitting #rag

2 min read

Tanishq Soni

Jun 28

Evaluating Large Language Models: The Overfitting Problem

#llm #evaluation #overfitting #rag

2 min read

Abdul Rehman

Jun 27

Building Evals That Don't Lie: How to Make AI Evaluation Reliable in Production

#ai #evaluation #production #llm

5 min read

Alex @ Vibe Agent Making

Jun 24

Our Quality Scores Were Precise, Useless, and Identical

#engineering #management #evaluation #codequality

8 min read

Saurav Bhattacharya

Jun 27

Who Grades the Grader? Your LLM Judge Is an Unvalidated Model in Production

#ai #evaluation #observability #testing

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.

DEV Community

# evaluation

One average eval score was hiding two different failure modes

Evaluating LLM Apps in Java

Evaluating LLM Apps in Python

The Mean Is Lying to You: Benchmarks Hide the Variance That Breaks Prod

Your LLM-as-judge has a position bias you are not measuring

Short-Circuit Your Agent Evals: Tier Order Is a Latency Budget, Not a Preference

Your AI judge might be reliable — and still be wrong

Reliable, and still wrong

Give Your Agent a Type Signature: Contract-First Output Beats a Smarter Judge

Your Model-as-Judge Doesn't Belong in the Hot Path

Evaluating Large Language Models: The Pitfall of Overfitting in RAG

Evaluating Large Language Models: The Overfitting Problem

Building Evals That Don't Lie: How to Make AI Evaluation Reliable in Production

Our Quality Scores Were Precise, Useless, and Identical

Who Grades the Grader? Your LLM Judge Is an Unvalidated Model in Production