Writing
Notes on AI evaluation
Practical articles on evaluating LLM apps, RAG systems and agents.
-
Hello world!
Welcome to WordPress. This is your first post. Edit or delete it, then start writing!
-
Why your LLM needs an eval set before launch
Demos prove that something can work. An eval set tells you how often it does — and whether your next change made it better…
-
LLM-as-judge: when to trust it
Using a model to grade another model scales beautifully — once you have checked that the judge agrees with the people who matter.
-
Evaluating RAG: beyond accuracy
A RAG answer can fail because the search missed, or because the model ignored what it found. Measure the two halves separately and you…