Screening LLM Judges Without Ground Truth: A Preliminary Evaluation of an AI Dental Hygiene Coach

Evaluating a fine-tuned coaching model with expert graders is slow and expensive, and an LLM judge cannot be trusted without ground truth. We present a staged pipeline that screens candidate LLM judges and student models before expert review, using explicit decision rules at each stage: judge output audit, similarity with hosted judges, bias checks, and decision robustness. We demonstrate it on…

Publisher's summary from arXiv cs.AI. Read the full article on arXiv cs.AI · More from arXiv cs.AI