pub async fn run_checks(
runner: impl Runner,
submission: &str,
checks: &[String],
) -> Vec<CheckOutcome>Expand description
Run each check against submission through runner, returning one
CheckOutcome per check, in order.
Effectful (§2.2). The submission is first run on its own: if it cannot
execute — a syntax error, or a launch failure — every check is
NotRun, because no check could have a verdict
(§3.3). This standalone gate is load-bearing: concatenating the submission
with a check and reading the combined exit code cannot tell a broken
submission from a violated check (and some interpreters, R among them, would
even splice an unterminated submission into the check). Once the submission
runs cleanly, each check is the submission followed by the check code-string;
the check passes when that program exits cleanly and fails otherwise.
Outcomes are element-wise — never folded into a single aggregate verdict.
With no checks there is nothing to classify, so the submission is not run at all — an LLM-only lesson (the common case) never pays for a subprocess here.