Module smevals_gen

Module smevals_gen 

Source
Expand description

smevals eval-dir generation: a pure translator from a lesson + eval suite to the file tree smevals run/build consume.

An eval dir is any directory with an eval.yaml, tasks/, configs/, and graders/ — the smevals v0.2.0 layout. generate_eval_dir produces that tree as (relative_path, contents) pairs: eval.yaml, configs/default.yaml, graders/default.yaml, and tasks/case-N.yaml (one per suite case, 1-based). It is pure (§2.1): no filesystem, no clock, no iteration-order randomness — identical inputs yield byte-identical output, and every value is emitted through an injection-proof YAML scalar discipline proven by round-trip tests (§1.3.1). The model id in configs/default.yaml is single-sourced from ProviderChoice::default_model(), never a divergent literal.

This module owns only the lesson+suite → smevals-dir translation. It is NOT the runner (AC-3), the LLM judge (AC-4), or the report command (AC-5); it merely emits the templates those later slices wire into. Script paths in the templates (scripts/smevals/run.sh, scripts/smevals/check_polarity.sh, scripts/smevals/judge_feedback.py) are emitted relative to the generated configs//graders/ file, as smevals resolves them relative to the file that names them.

YAML emission: serde-saphyr 0.0.27 is parse-only, so the emitter is hand-rolled. Hostile content (a submission containing : , &anchor, ---, or a leading - ) is emitted as a double-quoted scalar with every control character escaped — byte-exact under re-parse (serde_saphyr’s |+ chomping is broken and |- drops trailing newlines, so block scalars cannot round-trip arbitrary strings; the round-trip test in crates/core/tests/generate_eval_dir.rs pins this). Safe-by-construction values (slugs, verdict tokens, the pinned script paths) are plain scalars.

Enums§

GenError
Why an eval dir could not be generated.

Functions§

course_root_for
The course root for lesson_path: the nearest ancestor directory containing a blendtutor.toml manifest (the course boundary, per scaffold.rs).
generate_eval_dir
Generate the smevals eval-dir file tree for suite, tagged with lesson_id.
lesson_id_from_path
The lesson id for a lesson file: its stem, so lessons/foo.yaml is foo.
write_eval_dir
Write the generated eval dir into dir/.smevals/, where dir is the course root.