feat(olink): greenfield NPX-style proteomics simulator - #2
Merged
Conversation
Adds synthlab/olink.py, a small greenfield simulator for Olink-style
proteomics data (subject x protein -> NPX). As of 2026-04 there is no
widely-used open-source Olink simulator — the closest analogue,
MSstatsSampleSize, targets LC-MS/MS peptide intensities rather than
NPX/PEA, and OlinkAnalyze ships demo tables (npx_data1/npx_data2) but
no simulator.
Model: NPX[i,j] = mean[j] + plate_eff[i] + group_shift[i,j] + eps
- plate_eff ~ N(0, plate_effect_sd^2), 96 samples / plate
- group_shift from configurable per-protein {group -> delta} map
- eps ~ N(0, sd[j]^2)
Missingness
- mnar_lod: drops 80% of sub-LOD values (soft LOD, mirrors real Olink
behaviour where some sub-LOD values are still reported with
QC_WARN flags)
- mcar: uniform missing_rate drop on top
- mar: currently aliases mcar; full MAR deferred to follow-up
- none: skips missingness
Priors come from aggregate UKB-PPP statistics (Sun et al. 2023,
Nature — 2,923 proteins x ~54k participants) and OlinkAnalyze
npx_data1/npx_data2 demo tables: mean ~ 5 log2-units, sd ~ 0.6,
LOD ~ mean - 2*sd.
Ships a 50-protein preset (default_explore_3072_panel) subsetted from
UKB-PPP Explore 3072, plus parquet read/write helpers. Full NumPy-
style docstrings on every public function / class / method. 16 unit
tests cover determinism, group-effects mean shift, LOD drop,
MCAR overlay, plate variation, QC rate, parquet round-trip, schema
types, validation errors, and empty-frame edge cases.
Scope-deferred (future PR): full MAR missingness, multi-plate batch
effects beyond a flat per-plate intercept, realistic PEA dilution
noise model, and panel-version LOD bridging (Explore 3072 ↔ HT).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds notebooks/olink_demo.ipynb — a 24-cell walkthrough of synthlab.olink.simulate_olink_npx covering NPX distributions, LOD-driven missingness, plate batch effects, PCA/UMAP, a protein-protein correlation heatmap, a case-vs-control volcano plot, and a parquet round-trip. Runs end-to-end in ~20s on CPU with outputs embedded. Adds a notebooks/README.md index and a reproducible builder script (notebooks/_build_olink_demo.py). pyproject.toml picks up a new [viz] optional-dependency group (matplotlib, seaborn, scikit-learn, umap-learn) — not a hard dep; notebook users install via `pip install synthlab[viz]`.
This was referenced Apr 23, 2026
bschilder
added a commit
that referenced
this pull request
Apr 23, 2026
Ships synthlab/data/olink_disease_effects.csv — a curated, source-cited per-disease protein effect-size catalog covering 7 diseases × 43 rows (Alzheimer, BRCA_hereditary, CAD, CKD, Cancer_broad, IBD, T2D). Every row cites a real DOI (Sun et al. 2023 UKB-PPP, Williams et al. 2022 Sci Transl Med, Eldjarn et al. 2023 deCODE, Cohen et al. 2018 CancerSEEK, Dubin et al. 2023 Nat Comm CRIC, Guo et al. 2024 Nat Aging, Hu et al. 2025 Nat Comm UKB-PPP, Ahn et al. 2021 Cancers). Public API in synthlab.olink: - DiseaseEffectCatalog (frozen dataclass wrapping the loaded frame with .diseases(), .proteins_for(), .effects_for(noise_sd, seed)) - load_disease_effect_catalog(path=None) — importlib.resources-based ship-in-wheel loader Both re-exported from synthlab/__init__.py. Tests (tests/test_olink_disease_catalog.py, 15 tests): - schema, minimum disease coverage, DOI shape, defensible magnitudes - effects_for shape, KeyError paths, noise determinism / perturbation - end-to-end round-trip: catalog → effects_for → simulate_olink_npx, assert per-disease mean NPX shift matches catalog within 3 SE at 500 samples/arm × 44 rows Demo notebook (notebooks/olink_demo.ipynb) gains Section 10 — disease-conditional generation — with loading, inspection, 3-group cohort simulation (T2D/CAD/baseline, 300 samples each), top-5 grouped-bar chart, and a catalog-vs-empirical scatter. Rebuilt via notebooks/_build_olink_demo.py; executed outputs embedded. Cell count goes from 24 → 34. Docs: - README.md gains a "Disease-conditional effect catalog" subsection with per-disease row-range citations. - docs/olink_disease_catalog.md — full schema, unit-conversion guidance, SE guidance, evidence-strength rubric, PR checklist for adding a new disease. Ship-in-wheel plumbing: - synthlab/data/__init__.py + data/*.csv in [tool.setuptools.package-data] - .gitignore carve-out so synthlab/data/*.csv are tracked while other data/ and *.csv remain ignored. Stacks on PR #2 (feat/olink-npx-simulator). Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
synthlab/olink.py: a small greenfield simulator for Olink-style proteomics data (subject × protein → NPX) with LOD-driven missingness, configurable group effects, and per-plate batch effects.npx_data1,npx_data2) but no simulator.What shipped
synthlab/olink.py(~400 lines) with full NumPy-style docstrings:OlinkPanelConfig— frozen dataclass: proteins + per-proteinlod/mean/sd.OlinkSimConfig— frozen dataclass:n_samples,panel,group_effects,group_assignments,missingness,missing_rate,qc_warn_rate,plate_effect_sd,seed.simulate_olink_npx(config) -> pl.DataFrame: long-form frame withsample_id / protein_id / npx / qc_warning / group / plate_id.default_explore_3072_panel(mean=5, sd=0.6, lod=3): 50-protein preset drawn from UKB-PPP Explore 3072 (inflammation / cardiovascular / oncology markers).write_olink_parquet/load_olink_parquet: snappy-compressed I/O with schema validation.npx_data1/npx_data2NPX distributions.synthlab/__init__.py.tests/test_olink.pywith 16 unit tests (determinism, group-mean shift, LOD drop, MCAR overlay, plate variation, QC rate, parquet round-trip, schema types, validation errors, empty-frame).README.mdsection introducing the simulator.Model
Missingness:
mnar_lod: drops 80% of sub-LOD values (soft LOD — real Olink data sometimes still reports sub-LOD values with a QC warning).mcar: uniformmissing_ratedrop on top.mar: currently aliasesmcar.none: skips missingness.Scope-limits (explicit follow-ups)
The following are deferred to keep this PR small and testable:
"mar"aliases"mcar". A real MAR model would tie drop probabilities to the group label (e.g. drop rate doubles in "case" for the first 10% of proteins).olink.pyflat next to existing modules; a follow-up PR will restructure intosynthlab.{ehr,genomics,imaging,proteomics,nlp}subpackages.Test plan
pytest tests/test_olink.py -xvs— 16/16 pass locally in ~0.7sfrom synthlab import OlinkSimConfig, default_explore_3072_panel, simulate_olink_npxworks and returns a populated 6-column DataFrame.simulate_olink_npxpasses (python -m doctest synthlab/olink.py).tests/test_meds.pystill passes (30 passed, 1 skipped).Co-Authored-By: Claude Opus 4.7 (1M context) noreply@anthropic.com