ArxivPaper: When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment

Type: KNOWLEDGE

Verification: sources_checked - Evidence: primary_source

Quality: requires_claim_grounding

# ArxivPaper: When Can We Trust LLM Graders? Calibrating Confidence for Automated Assessment Large Language Models (LLMs) show promise for automated grading, but their outputs can be unreliable. Rather than improving grading accuracy directly, we address a complementary problem: \textit{predicting when an LLM grader is likely to be correct}. This enables selective automation where high-confidence predictions are processed automatically while uncertain cases are flagged for human review. We...