> For the complete documentation index, see [llms.txt](https://yahya-qanie.gitbook.io/yps-ai-agent/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://yahya-qanie.gitbook.io/yps-ai-agent/evaluation-and-benchmarking.md).

# Evaluation & benchmarking

Planned comparative testing; no completed results are reported here.

### Planned comparison

The plan is to run the same set of 200 YPS prompts across YPS AI, Gemini, Claude, and ChatGPT using their default configurations. The aim is to assess performance on questions relevant to the YPS field.

### Planned assessment

An AI judge is intended to score the full set. A human judge will independently assess a sample for accuracy, depth, and freshness. Comparing these judgments is necessary to assess whether automated scoring is reliable.

If human and AI scores diverge substantially, the AI judge's scores should not be assumed to measure quality reliably.

### Interpreting the method

The number of prompts alone does not establish statistical reliability. Results depend on question selection, model versions, available tools, scoring criteria, and the quality of human review.

### What evaluation cannot establish

A benchmark is evidence about performance under specified conditions. It does not guarantee correctness for every user, language, or country.