feat(meds): OMOP → MEDS conversion for ML-native EHR encoders - #1
Merged
Conversation
Adds synthlab.meds, a thin wrapper over the community meds_etl package (github.com/Medical-Event-Data-Standard/meds_etl). This lets SynthLab output feed directly into MEDS-native foundation models such as SMB-v1 (huggingface.co/standardmodelbio) and MOTOR. API: - MedsConvertConfig dataclass (omop_dir, meds_dir, backend, shards, proc, overwrite) — matches meds_etl_omop's CLI surface. - convert_omop_to_meds(config): runs meds_etl_omop as subprocess, surfaces stderr on failure, refuses to overwrite unless asked. - load_meds_events(meds_dir, subject_ids=None): polars-native reader for the sharded parquet output, sorted by (subject_id, time). - get_meds_cache_dir / get_meds_info / print_meds_info — mirrors the pattern every other synthlab submodule uses. Pipeline: SyntheaRunner.run() -> Synthea CSVs convert_synthea_to_omop() -> OMOP v5.4 convert_omop_to_meds() -> MEDS parquet (downstream: SMB-v1 / MOTOR encoder) Install: `pip install synthlab[meds]` (adds meds_etl + meds pins). Infra: - New tests/ dir + test_meds.py (15 tests, all passing on py3.11): cache-dir, info dict, config validation, error paths, shard reading, subject filtering, meds_etl entry-point registration. - New .github/workflows/test.yml runs pytest on every PR + push. - pyproject.toml gains [meds] + [tests] optional extras and pulls them into [all]. - README gets a "MEDS conversion" section with quickstart. No breaking changes to existing APIs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
synthlab.meds, a thin wrapper over the communitymeds_etlpackage so SynthLab output (Synthea → OMOP CDM) can feed directly into MEDS-native foundation models like SMB-v1 and MOTOR.Pipeline
Public API
MedsConvertConfigdataclass — mirrors themeds_etl_omopCLI surface (backend, num_shards, num_proc, overwrite)convert_omop_to_meds(config)— invokesmeds_etl_omopas a subprocess, surfaces stderr on failure, refuses to clobber existing output unless askedload_meds_events(meds_dir, subject_ids=None)— polars-native reader for the sharded parquet, sorted by(subject_id, time); cohort filter pushed into the scanget_meds_cache_dir()/get_meds_info()/print_meds_info()— matches the pattern every other synthlab submodule usesInstall
pip install synthlab[meds](new optional extra; pullsmeds_etl>=0.3+meds>=0.4).Infra
tests/directory +test_meds.py— 15 tests covering cache-dir, info dict, config validation, all error paths (ImportError, FileNotFoundError, FileExistsError), shard reading, subject filtering, andmeds_etl_omopconsole-script registration.github/workflows/test.yml— runs pytest on every PR + push tomain(previously no CI existed)pyproject.tomlgets new[meds]and[tests]extras (rolled into[all])Test plan
pytest tests/passes locally (15/15) on Python 3.11import synthlab.medsclean in fresh envconvert_omop_to_medsraises clear errors for missingmeds_etl, missing OMOP dir, and existing output dir withoutoverwrite=Trueload_meds_eventsround-trips a synthetic parquet shard + respectssubject_idsfilterMotivation
Enables downstream work in smb-protopheno where each cohort's synthetic EHR needs to be encoded by SMB-v1 as part of a joint genomics + EHR model. Without this, every consumer of SynthLab's output has to re-implement the OMOP → MEDS ETL.
🤖 Generated with Claude Code