Repository navigation
docs: rerun PK's batch BLQ cells and address the review of the comparisons - #684
Open
billdenney wants to merge 10 commits into
Open
billdenney wants to merge 10 commits into
billdenney wants to merge 10 commits into
Conversation
…ethods The sparse AUC vignette places PKNCA's methods beside the serial-sacrifice and flexible-design estimators of Wolfsegger and Jaki (AUC to infinity in 2005, the complete/batch/serial AUC to the last time in 2009, the full parameter set in 2012) and the comparison of methods for samples below the limit of quantification by Barnett, Geys, Jacobs, and Jaki (2021). The 2005 paper fits lambda.z to time points after the trapezoidal sum so the two parts are independent and names the covariance between them as open; the vignette says that PKNCA's delta method uses the same samples for both and that covariance. The help of var_sparse_auc() and sparse_bootstrap() and the sparse AUC-to-infinity roxygen carry the matching references, the author names join the wordlist, and NEWS records the change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The simulation-estimation studies behind the sparse AUC to infinity vignette move from six top-level scripts in data-raw/ into a targets pipeline in data-raw/sparse-simulation/: the design, the data sets of each scenario (static branches over the scenarios, seeded per data set), each study's results, the summaries, and the vignettes/v24-*.rds files as the final targets. The scripts' logic and seeds are kept; at a small number of replicates the pipeline gives results identical to the scripts, replicate by replicate and summary by summary, so the stored results do not change. The pipeline also has the new comparisons: the PK package's asymptotic and bootstrap-t intervals, the Wolfsegger and Jaki (2005) estimator written from the paper, a bootstrap-t interval on PKNCA's delta-method standard error, and the BLOQ package's treatments of BLQ samples. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With BLQ samples given as zero, PK::nca() for a batch design stops with an error after oral dosing, and after IV bolus dosing it often returns a standard error of the AUC to infinity that is not finite or is essentially zero. Both now count as unusable: the data set is rerun with the BLQ samples omitted, and a data set with no usable result is a failure rather than an interval of zero width. The serial-sacrifice results are unchanged (no serial data set gave an unusable result), and only the eight batch scenarios with BLQ samples were rerun. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eline vignettes/v24-sparse-auc-to-infinity-comparisons.rds holds the coverage, width, and bias of PKNCA's, the PK package's, and the Wolfsegger and Jaki (2005) intervals on the serial-sacrifice (three and four animals per time) and batch scenarios, and of PKNCA's bootstrap-t over the number of animals per time. vignettes/v24-sparse-auc-to-infinity-blq.rds holds the BLOQ package's treatments of BLQ samples on the scenarios with BLQ samples. The README of data-raw/sparse-simulation/ says how to run the pipeline, what it costs, and how to add a scenario or an estimator. The comparisons took 10.5 hours with 6 workers on an 8-core laptop, and the rerun of the batch scenarios with BLQ samples 2.3 hours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sparse AUC vignette gains three results sections and their findings: PKNCA's intervals against the Wolfsegger and Jaki (2005) estimator and the PK package (Wolfsegger and Jaki 2009) on the serial-sacrifice scenarios with three and four animals per time; the PK package's AUC, AUMC, MRT, clearance, and Vss intervals against PKNCA's on the batch scenarios; and the BLOQ package's treatments of BLQ samples (Barnett et al. 2021) against zero substitution and the Tobit half-life. The profile over animals per time gains a bootstrap-t interval on PKNCA's delta-method standard error, and the Discussion gains a subsection on what PKNCA has that the published methods do not and what it does not adopt from them. The Summary and recommendations and the Introduction describe the new results, and the vignette points to the pipeline's README for how to run the simulations. PK and BLOQ join Suggests, at Bill Denney's request, for these comparisons; the package does not call them, and the vignette reads the stored results. New words go in inst/WORDLIST, and the references gain Tardivon et al. (2023) and the PK and BLOQ packages. The README's run times are corrected to the measured 60 core-hours. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The vignette reads the fields, summary columns, and interval names of its seven result files by name. The test fails when a pipeline change renames or drops one, when a version field is missing or malformed, or when a result file is missing, rather than letting the vignette build fail. It needs the package source and skips without it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…swer gates PK's batch-design code replaces a NaN variance with 1e-08, so the reported standard error is sqrt(1e-08/n). The earlier usability rule (standard error above 1e-6 times the estimate) let that placeholder pass for estimates below about 50, so in the IV bolus batch scenarios with BLQ samples many data sets kept their zeros and were scored with a standard error near zero. PK's batch function cannot take a zero at a time where other samples are positive, so every batch data set with BLQ samples is now given to PK with them omitted, and every row (AUC, AUMC, MRT, half-life, clearance, Vss) whose standard error is the placeholder or is not finite is a failure. The serial-sacrifice results are unchanged. The pipeline gains: - known-answer tests in data-raw/sparse-simulation/tests/testthat/: a literal loop transcription of equations 2 to 7, 15, 3, and 20 of Wolfsegger and Jaki (2005) against wj2005_estimate(), the exact answer with no variability, and hand-made intervals with known coverage, width, and bias for summarize_intervals(). They run as the target known_answers, and the comparisons' result file is not written unless they pass. - the large-sample bias of the 2005 estimate and PKNCA's (2,000 animals per time, 20 data sets of each serial scenario without BLQ samples), as targets, stored in the comparisons' result file. - SIM_OUTPUT_DIR, so a trial run writes its result files elsewhere rather than over vignettes/. The README describes the new settings and tests, and states the run time of the whole pipeline (about 320 core-hours) without ambiguity. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bill Denney asked that PK and BLOQ be used only for the precomputation in data-raw/, not listed as package dependencies. Neither the package, its tests, nor the vignette calls them; the vignette reads the stored results. The NEWS bullet and the pipeline's README say so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…isons The eight batch scenarios with BLQ samples were rerun with every data set that has BLQ samples given to PK with them omitted, and PK's placeholder standard errors counted as failures. Only PK's rows of those cells change; in the two-compartment IV bolus scenarios, PK's asymptotic interval for the AUC to infinity now covers 95.8 and 91.2 percent rather than 47.8 and 60.0 percent, which measured the placeholder. The vignette: - describes PK's batch behaviour with zeros correctly (an error after oral dosing, a placeholder variance after IV bolus dosing) and rewrites the batch findings with the new results; - takes the large-sample bias of the 2005 estimate from the pipeline instead of a typed number (1.2 to 2.6 percent with fast elimination, -2.4 to -0.7 percent with slow, against PKNCA's 2.6 to 4.6 and -1.6 to -0.8); - says that coverage is among the data sets whose interval could be calculated, with the failure rates and the largest effect of counting a failure as a miss; - says the 2005 bootstrap-t covered "about as well" without BLQ samples, and cites Jaki and Wolfsegger (2011), PK's recommended citation. The result-file test checks the new large-sample summary and says that it skips under R CMD check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bill Denney's review: the bootstrap-t on the delta-method standard error keeps the same point estimate, so the same bias, and its better coverage of the AUMC to infinity comes from the interval, not from a better estimate. The findings, the Summary, and the Extensions subsection now say so with the numbers that show it: the AUMC bias is the same in the delta and bootstrap-t rows (-17.1 percent with 4 animals per time, -15.9 with 100); with 4 animals the bootstrap-t interval is 1.54 times as wide (coverage 79.8 to 84.4 percent); with 100 it is about as wide (1.05 times), and its asymmetric quantiles move it up, so the share of intervals wholly below the truth falls from 69.6 to 55.4 percent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The eight batch scenarios with BLQ samples were rerun with every data set
that has BLQ samples given to PK with them omitted, and PK's placeholder
standard errors counted as failures. Only PK's rows of those cells
change; in the two-compartment IV bolus scenarios, PK's asymptotic
interval for the AUC to infinity now covers 95.8 and 91.2 percent rather
than 47.8 and 60.0 percent, which measured the placeholder.
The vignette:
oral dosing, a placeholder variance after IV bolus dosing) and rewrites
the batch findings with the new results;
instead of a typed number (1.2 to 2.6 percent with fast elimination,
-2.4 to -0.7 percent with slow, against PKNCA's 2.6 to 4.6 and -1.6 to
-0.8);
calculated, with the failure rates and the largest effect of counting a
failure as a miss;
and cites Jaki and Wolfsegger (2011), PK's recommended citation.
The result-file test checks the new large-sample summary and says that it
skips under R CMD check.
Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com