eval/data/01_raw/datasets/{train,val} (~5.9 GB, ~97k files) is a dep of evaluate@train/@val but is not tracked by any .dvc file and not produced by any eval stage — it is refreshed by hand-rsyncing train/data/01_raw/datasets/ (now documented in docs/runbooks/retrain-on-new-dataset.md step 3, but still manual).
Failure modes:
- Skip or mistime the rsync and eval silently scores the new model against the previous release's truncated data; the run succeeds and the lock records a self-consistent hash, so nothing distinguishes it from a correct run.
dvc push never uploads these files, so a fresh clone / CI cannot reproduce the eval pipeline without cloning train and re-running truncate.
Proposed fix: mirror the pattern already used for model.zip one bullet down — a frozen local dvc import of train's truncate outputs plus an update-datasets make target, making the dep pushable, pullable, and reviewable. Noticed in the PR #65 review; kept out of that PR to avoid scope creep.
eval/data/01_raw/datasets/{train,val}(~5.9 GB, ~97k files) is a dep ofevaluate@train/@valbut is not tracked by any.dvcfile and not produced by any eval stage — it is refreshed by hand-rsyncingtrain/data/01_raw/datasets/(now documented indocs/runbooks/retrain-on-new-dataset.mdstep 3, but still manual).Failure modes:
dvc pushnever uploads these files, so a fresh clone / CI cannot reproduce the eval pipeline without cloning train and re-runningtruncate.Proposed fix: mirror the pattern already used for
model.zipone bullet down — a frozen localdvc importof train's truncate outputs plus anupdate-datasetsmake target, making the dep pushable, pullable, and reviewable. Noticed in the PR #65 review; kept out of that PR to avoid scope creep.