feat(alignment): add cross-configuration framework#230
Open
CyberSecurityErial wants to merge 8 commits into
Open
feat(alignment): add cross-configuration framework#230CyberSecurityErial wants to merge 8 commits into
CyberSecurityErial wants to merge 8 commits into
Conversation
CyberSecurityErial
requested review from
Flink-ddd,
KJLdefeated,
bitborne and
inaniloquentee
as code owners
July 19, 2026 00:43
📝 WalkthroughWalkthroughIntroduces a versioned cross-configuration framework for strict experiment planning, semantic operator resolution, runtime materialization, paired rollout/training scoring, fixed-threshold comparison, append-only artifacts, resume validation, CLI execution, and CPU smoke testing. ChangesCross-configuration alignment
Estimated code review effort: 5 (Critical) | ~120 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 10
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
rl_engine/executors/stateless_executor.py (1)
259-271: 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick winValidate the injected operator’s exact device and dtype.
The assignment into
resultsilently casts a wrong dtype, allowing a backend to violate its declared output contract while rank validation still passes.Proposed validation
if shifted_logps.shape != shifted_labels.shape: raise ValueError( "selected_logprob_fn output shape must match selected token IDs, got " f"{tuple(shifted_logps.shape)} and {tuple(shifted_labels.shape)}" ) + if shifted_logps.device != logits.device: + raise ValueError("selected_logprob_fn output device must match logits") + if shifted_logps.dtype != output_dtype: + raise ValueError("selected_logprob_fn output dtype must match output_dtype")🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@rl_engine/executors/stateless_executor.py` around lines 259 - 271, Extend validation in the shifted_logps handling near result construction to require that the injected operator output matches the expected device and dtype exactly, in addition to the existing tensor and shape checks. Raise a clear validation error before assigning to result, preserving the existing assignment only for correctly typed and colocated tensors.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@rl_engine/alignment/cross_config/__init__.py`:
- Around line 28-40: Update the __all__ list in the cross_config package to
satisfy Ruff RUF022 by sorting the lowercase exports alphabetically: place
build_execution_plan before compare_score_artifacts, followed by load_config,
while leaving the other exports unchanged.
In `@rl_engine/alignment/cross_config/_provenance.py`:
- Around line 51-56: Update the status ordering in the precedence tuple used by
the provenance aggregation logic so MaterializationStatus.UNOBSERVABLE appears
before MaterializationStatus.FALLBACK. Preserve the remaining precedence order
and align this behavior with RuntimeTools._aggregate_status, ensuring mixed
unobservable and fallback results aggregate to UNOBSERVABLE.
In `@rl_engine/alignment/cross_config/_resume.py`:
- Around line 58-75: Update the resume validation flow to canonically validate
every required artifact before returning True, including materialized.json and
the complete comparison.json and COMPLETE contents. Compare persisted
materialization with materialization.materialized_case, reconstruct and compare
the full alignment result and completion summary, or validate hashes for all
artifacts sealed by COMPLETE. Ensure any missing, mutated, or inconsistent
artifact causes resume validation to return False.
In `@rl_engine/alignment/cross_config/comparison.py`:
- Around line 110-147: Sanitize non-finite inactive values before constructing
TokenComparisonArtifact in the comparison flow. Preserve the existing
active-value validation, then replace inactive entries in rollout_logprobs,
training_logprobs, and absolute_diff with finite zero values before persistence,
while keeping active values unchanged.
In `@rl_engine/alignment/cross_config/planner.py`:
- Around line 201-230: Add a fixed framework case-count limit to the
plan-generation logic around normalize_requested, intervention expansion, and
pairwise product generation. Check the count before appending each requested
case, stop immediately when the limit would be exceeded, and raise the planner’s
structured PLAN_TOO_LARGE error rather than allowing unbounded accumulation;
apply the same guard to both single-intervention and pairwise paths.
In `@rl_engine/alignment/cross_config/runtime.py`:
- Around line 312-333: Extend the validation around expected, descriptors, and
applications to reject every normalized path missing from descriptors,
preventing it from bypassing descriptor checks. For APPLIED applications,
require actual to equal materialized, and for non-derived knobs require
materialized to equal the normalized request. Add these failures to problems so
require_executable() cannot trust invalid adapter status.
In `@rl_engine/alignment/cross_config/schema.py`:
- Around line 525-526: Update the fixed_threshold validation in the relevant
schema class to reject all non-finite values, including NaN and positive
infinity, while retaining the existing non-negative requirement. Use an
is-finite check alongside the current comparison before raising the existing
ValueError.
In `@rl_engine/kernels/gtest/tolerance.py`:
- Around line 29-35: Update the threshold handling in the function containing
_normalize_dtype_name and load_contract so values["atol"] is converted and
validated as a real, finite, non-negative number, rejecting booleans, NaN,
infinity, negatives, and malformed entries. Convert any invalid threshold input
into ValueError while preserving the existing missing-dtype ValueError behavior.
In `@rl_engine/kernels/semantic_registry.py`:
- Around line 282-285: Update the topology matching in the strict resolution
flow around _supports so missing required topology fields do not match
accidentally. Require every explicitly declared descriptor topology key,
including world_size, tensor/context parallelism, and sharding, to be present
and matched; only allow an empty or incomplete requirement set when the
descriptor declares the wildcard "*".
In `@tests/test_cross_config_runtime.py`:
- Line 445: Update the pytest.raises match patterns at the affected tests to
escape the dots in “training.attention_backend” (and the corresponding pattern
near the second occurrence), so pytest treats the dotted path literally and
satisfies RUF043.
---
Outside diff comments:
In `@rl_engine/executors/stateless_executor.py`:
- Around line 259-271: Extend validation in the shifted_logps handling near
result construction to require that the injected operator output matches the
expected device and dtype exactly, in addition to the existing tensor and shape
checks. Raise a clear validation error before assigning to result, preserving
the existing assignment only for correctly typed and colocated tensors.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: eaaa134d-3ec1-4de6-bb80-2bb55310aa31
⛔ Files ignored due to path filters (1)
docs/assets/ws2-cross-config-before-after.pngis excluded by!**/*.png
📒 Files selected for processing (41)
.github/workflows/ci.yml.gitignoredocs/design/cross_config_implementation_report.mddocs/design/cross_config_logprob_drift_contract.mddocs/design/ws2_cross_config_logprob_drift_contract.mdexamples/cross_config_s0_cpu_smoke.jsonexamples/cross_config_s1_distributed_smoke.jsonexamples/cross_config_s2_vllm_tp_vs_fsdp.jsonexamples/cross_config_s3_qwen3_8b_tp4_cp4_bf16.jsonpyproject.tomlrl_engine/alignment/cross_config/__init__.pyrl_engine/alignment/cross_config/__main__.pyrl_engine/alignment/cross_config/_execution.pyrl_engine/alignment/cross_config/_json.pyrl_engine/alignment/cross_config/_provenance.pyrl_engine/alignment/cross_config/_resume.pyrl_engine/alignment/cross_config/artifacts.pyrl_engine/alignment/cross_config/comparison.pyrl_engine/alignment/cross_config/config.pyrl_engine/alignment/cross_config/execution_plan.pyrl_engine/alignment/cross_config/operators.pyrl_engine/alignment/cross_config/planner.pyrl_engine/alignment/cross_config/runner.pyrl_engine/alignment/cross_config/runtime.pyrl_engine/alignment/cross_config/schema.pyrl_engine/alignment/testing/__init__.pyrl_engine/alignment/testing/cpu_cross_config.pyrl_engine/alignment/testing/smoke_ops/SMOKE_OPERATORS.mdrl_engine/alignment/testing/smoke_ops/__init__.pyrl_engine/alignment/testing/smoke_ops/smoke_only_logp_offset.pyrl_engine/alignment/testing/smoke_ops/smoke_only_logp_reference.pyrl_engine/executors/stateless_executor.pyrl_engine/kernels/gtest/tolerance.pyrl_engine/kernels/registry.pyrl_engine/kernels/semantic_registry.pytests/test_cross_config_cli.pytests/test_cross_config_contract.pytests/test_cross_config_runner.pytests/test_cross_config_runtime.pytests/test_stateless_executor.pytests/test_tolerance_contract.py
💤 Files with no reviewable changes (1)
- docs/design/ws2_cross_config_logprob_drift_contract.md
CyberSecurityErial
force-pushed
the
feat/cross-config-alignment
branch
from
July 19, 2026 01:19
35073a1 to
1c4457e
Compare
CyberSecurityErial
changed the title
Flink-ddd
requested review from
EthanZero2Hero,
a-kaa,
maxiaosong1124 and
zhangj1an
and removed request for
bitborne
July 19, 2026 06:56
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a compact cross-configuration alignment framework with a CPU-only executable smoke path and explicit extension boundaries for future production adapters.
Functional modules
Scope and claim boundary
The executable path validates framework plumbing with a synthetic CPU model. It does not claim production vLLM, FSDP, tensor/context-parallel, distributed, or accelerator numerical alignment. The larger named scenarios are reproducible planning inputs until production adapters expose verified injection and read-back hooks.
Planned cases:
Validation
60 passed.418 passed, 242 skipped.git diff --checkpass.The CPU suite excludes the two Triton-required collection modules; Gloo and POSIX shared-memory tests were run with host resources available.
Review order
feat(kernels): add semantic operator catalogfeat(alignment): define cross-configuration contracts and plansfeat(alignment): add runtime materialization and scoring bridgefeat(alignment): add execution and artifact primitivesfeat(alignment): add provenance-aware paired executionfeat(alignment): add CPU smoke workflow and CLItest(alignment): add focused cross-configuration coveragedocs(alignment): document cross-configuration workflowRelated: #83, #108, #111, #222.
Summary by CodeRabbit
New Features
Bug Fixes
Documentation