pub async fn run_eval(
lesson: &Lesson,
suite: &EvalSuite,
provider: ProviderChoice,
base_url_override: Option<&str>,
case: Option<usize>,
) -> Result<EvalReport, EvalRunError>Expand description
Run every case in suite through the same pipeline run uses and score each
verdict against its expected polarity — or, when case is Some(n), only
the one 1-based case n.
The thin effectful shell over the pure scorer (§2.4): for each case it runs
the submission through run_lesson — execute, grade, ask provider for a
verdict — then pairs the verdict’s polarity with the expected one as a
CaseResult. Driving the same run_lesson is what keeps the evaluated
feedback identical to the shipped feedback (§3.2). An out-of-range case
selection is rejected up front with EvalRunError::CaseOutOfRange — never
clamped — before any case runs; a run failure stops scoring and names the
offending case (EvalRunError::Run). base_url_override points the
provider at a stub in tests, exactly as run does, and is None in
production.