Repository navigation

Introduce `model_compare()` for model comparison with support of new predictive measures - #380

Merged

florence-bockting merged 90 commits into

pred_measurefrom

integrate-loo_compare

Oct 5, 2026

Commits on Jul 7, 2026
- refactor: update loo_compare to work with loo_pred_measure
  Florence Bockting
  committed
  89d5e9a
- refactor: update pred_measure API for integration with loo_compare()
  Florence Bockting
  committed
  7ce2d50
- tests: new tests for loo_compare with loo_pred_measure
  Florence Bockting
  committed
  7394ba3
- docs: update documentation and glossary
  Florence Bockting
  committed
  494d24f
- docs: update documentation
  Florence Bockting
  committed
  9611656
- chore: update developer-notes
  Florence Bockting
  committed
  a3019e3
Commits on Jul 8, 2026
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  3e1bf93
- refactor: clean-up refactoring and adjust for renamed arg
  Florence Bockting
  committed
  06d6281
- fix: update missing argument and corresponding docs
  Florence Bockting
  committed
  b46ff58
- chore: update NEWS.md
  Florence Bockting
  committed
  f1fcaf0
Commits on Jul 16, 2026
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  e17e46d
Commits on Aug 12, 2026
- refactor: update computation of SE for loo_compare with pred_measure_loo
  Florence Bockting
  committed
  abc74b5
Commits on Aug 25, 2026
- refactor: create model_compare and workout design
  Florence Bockting
  committed
  faa27c5
Commits on Aug 27, 2026
- update model_compare() functionality
  Florence Bockting
  committed
  3e078aa
Commits on Aug 28, 2026
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  8897e1f
- refactor: split model_compare.R and drop duplicated loo_compare internals
  
  Florence Bockting
  and
  claude
  committed
  1eb66de
- fix: restore the simplify argument to print.compare.loo
  
  Florence Bockting
  and
  claude
  committed
  166dab6
- review: update the model_compare function
  Florence Bockting
  committed
  c0c1fc0
- update: deprecate loo_compare but maintain backwards comptability
  Florence Bockting
  committed
  c594818
- review: code refactoring review
  Florence Bockting
  committed
  aa30e54
Commits on Aug 29, 2026
- docs: update function documentation to roxygen2 8.0.0
  Florence Bockting
  committed
  b1f0b32
- docs: update glossary to match loo_compare and model_compare
  Florence Bockting
  committed
  63c16a8
- fix: throw deprecation warning for loo_compare only once per session.
  Florence Bockting
  committed
  d874044
- fix: remove old 'estimates_only' label
  Florence Bockting
  committed
  9e930e3
- refactor: avoid duplicate computation
  Florence Bockting
  committed
  4fe4b0d
- docs: add loo_compare and model_compare to seealso statement
  Florence Bockting
  committed
  88890b6
- refactor: add warning for disagreeing measure name in attribute and list
  Florence Bockting
  committed
  51b3c9c
- review: minor style corrections and doc adjustment
  Florence Bockting
  committed
  81ae8eb
- review: removed dead code
  Florence Bockting
  committed
  fed3433
- vignette: update tutorial on model comparison
  Florence Bockting
  committed
  585b77c
- chore: update warning messages
  Florence Bockting
  committed
  3f63357
- review: update function docs and inline small helper
  Florence Bockting
  committed
  5307f23
Commits on Aug 31, 2026
- Merge branch 'pred_measure' into integrate-loo_compare
  Florence Bockting
  committed
  795cd4e
- tests: shrink the model-comparison fixture
  Florence Bockting
  committed
  69346f3
- print: accept simplify on the pred_measure path
  Florence Bockting
  committed
  e52ab8e
- model_compare: accept named models in dots
  Florence Bockting
  committed
  d55d9bf
- internal: share the k-fold K-mismatch warning
  Florence Bockting
  committed
  0f830e3
- internal: drop model_compare_checks.psis_loo_ss_list()
  Florence Bockting
  committed
  769b37a
- internal: drop model_compare_matrix.psis_loo_ss_list()
  Florence Bockting
  committed
  b80c946
- docs: correct the Sivula et al. year in the glossary
  Florence Bockting
  committed
  ca8da41
- pred_measure: keep a renamed measure's own orientation
  Florence Bockting
  committed
  47af38e
- docs: remove draft scaffolding from the comparison article
  Florence Bockting
  committed
  99d71a2
- docs: let the comparison article fit its own models
  Florence Bockting
  committed
  0e07f63
- docs: point the custom-measure chunks at the new fits
  Florence Bockting
  committed
  4317ce7
- docs: rename the comparison article to model-comparison
  Florence Bockting
  committed
  0c1de0c
- docs: correct the article path in the model_compare links
  Florence Bockting
  committed
  c66e1e2
- internal: drop the vignette fits from the data generator
  Florence Bockting
  committed
  5f6bcb9
- docs: give the two remaining articles their real vignette title
  Florence Bockting
  committed
  6dbd05d
Commits on Sep 1, 2026
- model_compare: return the default object for subsampled loo
  Florence Bockting
  committed
  91bfdba
- model_compare: let simplify = FALSE follow measures
  Florence Bockting
  committed
  6621d2d
- model_compare: name the fold count and the test size
  Florence Bockting
  committed
  0944a95
- vignettes: update model-comparison tutorial with other pred_measure family types
  Florence Bockting
  committed
  0140dc1
Commits on Sep 9, 2026
- refactor: allow attribute measure_se_diff for custom measures
  Florence Bockting
  committed
  67f4d12
- docs: update developer-notes with review comments from Jonah
  Florence Bockting
  committed
  e339f48
Commits on Sep 16, 2026
- refactor: add user facade custom_measure() and remove custom_se_fn parameter from model_compare()
  Florence Bockting
  committed
  4c5a1a8
Commits on Sep 19, 2026
- Merge remote-tracking branch 'origin/pred_measure' into integrate-loo_compare
  Florence Bockting
  committed
  aa7f286
Commits on Sep 21, 2026
- docs: update developer-notes wrt custom_measure
  Florence Bockting
  committed
  6971e13
- fix: model_compare throws warning when kfold_pred_measure use different folds; update vignette example
  Florence Bockting
  committed
  8815998
- fix: pass moment_match loss into pred_measure
  Florence Bockting
  committed
  eec2140
- fix: use separate digits for measures in print method
  Florence Bockting
  committed
  dcf5fde
- fix: update snapshots
  Florence Bockting
  committed
  ba29be1
Commits on Sep 25, 2026
- refactor: remove the 'rank_by' argument for model_compare
  Florence Bockting
  committed
  c5af96b
- fix: check whether pothoc measures have been used and warn for measures other than elpd, mlpd, and ic
  Florence Bockting
  committed
  9ffff9c
Commits on Sep 28, 2026
- fix: remove diagnostic flags for measures other than elpd
  Florence Bockting
  committed
  dd2ac81
- docs: rebuild loo_pred_measure.Rd for the post-hoc correction note
  Florence Bockting
  committed
  e372858
- feat: mark and explain flipped measure signs in print
  Florence Bockting
  committed
  36795c2
- docs: move model_compare design notes to design-discussions
  Florence Bockting
  committed
  999cfa7
- docs: shorten NEWS entries for model_compare
  Florence Bockting
  committed
  de54993
- docs: update model_compare vignette
  Florence Bockting
  committed
  cd1085d
- Merge remote-tracking branch 'origin/pred_measure' into integrate-loo_compare
  Florence Bockting
  committed
  954bdb1
Commits on Sep 29, 2026
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  390eeb1
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  fbe6e88
- tests: make the test fixtures smaller for CRAN
  Florence Bockting
  committed
  ee7b99b
- model_compare: rank by elpd, else by the first shared measure in alphabetical order
  Florence Bockting
  committed
  fd427d9
- model_compare: drop complexity columns by exact name, keep custom p_ measures
  Florence Bockting
  committed
  f7a7b4d
- fix: define missing fold in example of model_compare
  Florence Bockting
  committed
  a509447
- docs: remove measure-ordering code comment from example
  Florence Bockting
  committed
  102a76b
- chore: update deprecation warning to not print internal function and remove unnecessary line break
  Florence Bockting
  committed
  d03261f
Commits on Sep 30, 2026
- print.compare.loo: warn when is given for 'loo' comparisons
  Florence Bockting
  committed
  18fd34b
- docs: measures applies to all pred_measure comparisons
  Florence Bockting
  committed
  7573b3d
- docs: change title of model-comparison.Rmd and change link in function documentation in model_compare
  Florence Bockting
  committed
  07d68c8
- docs: update the description of model-compare.Rmd in 'See also' section in model_compare
  Florence Bockting
  committed
  d958069
- docs: update EVAL parameter in vignette
  Florence Bockting
  committed
  2af1cd1
- docs: update vignette section to 'Basic usage'
  Florence Bockting
  committed
  1ac844b
- fix: point custom measure errors to custom_measre()
  Florence Bockting
  committed
  fa986da
- docs: point custom measure declarations to custom_measure()
  Florence Bockting
  committed
  706bc0f
- fix: name the ranking reference when more than four measures are compared
  Florence Bockting
  committed
  88afda5
- Merge branch 'pred_measure' into integrate-loo_compare
  florence-bockting
  authored
  0bd95d6
Commits on Oct 5, 2026
- fix: remove test_that from inner test
  Florence Bockting
  committed
  c9f72c6
- fix: remove attributes names pointing to older custom_measure design
  Florence Bockting
  committed
  2041008

Commits

Commits on Jul 7, 2026

Commits on Jul 8, 2026

Commits on Jul 16, 2026

Commits on Aug 12, 2026

Commits on Aug 25, 2026

Commits on Aug 27, 2026

Commits on Aug 28, 2026

Commits on Aug 29, 2026

Commits on Aug 31, 2026

Commits on Sep 1, 2026

Commits on Sep 9, 2026

Commits on Sep 16, 2026

Commits on Sep 19, 2026

Commits on Sep 21, 2026

Commits on Sep 25, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026

Commits on Sep 30, 2026

Commits on Oct 5, 2026