Independent run

Independence has two components: who executed the evaluation, and whether the settings were held constant across models. A third party running every model through the same harness and grader produces rows that can be compared to each other. Vals AI runs every model on its own boards, 85 models on MedCode and 84 on MedScribe. Microsoft Research runs HealthAgentBench across agent harnesses rather than bare models, and Copilot, Microsoft's own harness, does not top it.

Independence is not the same as disinterest. HealthAdminBench is run by Kinetic Systems' research team, independent of the model vendors, and published by a company that sells healthcare administration automation. Health Optimization Bench addresses the authorship version of the problem structurally: tasks are written by one model family, audited by families that did not author them, and graded blind by a panel of independent families, with 95 percent bootstrap confidence intervals accompanying every score.

Independent runs cost more and lag releases. Boards that run everything themselves refresh on a schedule rather than on launch day, so a new model can be absent for weeks, and academic tables often freeze their model set at submission. MedHELM, at version 5.0.0 and last updated May 14, 2026, refreshes on roughly a quarterly cadence. The tradeoff is coverage against comparability, and a reader who wants both has to accept a delay.

Basis labels of independent-run, vendor-reported, official-leaderboard, and mixed are recorded per benchmark at clinicalbenchmarks.ai, which also holds the current standings. The dedicated boards are healthbenchprofessional.com, healthbenchhard.ai, and healthoptimizationbench.com.