The Judge Can Debias Itself: Awareness and Self-Debiasing for Reliable LLM-as-a-Judge

LLM-as-a-Judge systems are increasingly employed for automated evaluation, yet remain susceptible to systematic biases that undermine their reliability. We demonstrate that the judge can effectively debias itself without additional training. We propose two lightweight strategies: bias-aware prompting, which explicitly alerts the judge to potential target biases prior to evaluation, and…

Publisher's summary from arXiv cs.AI. Read the full article on arXiv cs.AI · More from arXiv cs.AI