[API Compatibility] Add alias for SGD, Adagrad, AdamW#79284
[API Compatibility] Add alias for SGD, Adagrad, AdamW#79284algorithm1832 wants to merge 15 commits into
SGD, Adagrad, AdamW#79284Conversation
|
这个冲突了。其他PR做了alias精简处理，可能要重新处理下别名这块，必要的话重新新增文件。 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## develop #79284 +/- ##
==========================================
Coverage ? 97.56%
==========================================
Files ? 5
Lines ? 41
Branches ? 0
==========================================
Hits ? 40
Misses ? 1
Partials ? 0 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
| from . import lr_scheduler # noqa: F401 | ||
|
|
||
|
|
||
| class Adagrad(PaddleAdagrad): |
There was a problem hiding this comment.
这个会使paddle.optim.adagrad.Adagrad调用成原来的吧，需要新增文件
| ) | ||
|
|
||
|
|
||
| class SGD(PaddleSGD): |
There was a problem hiding this comment.
paddle.optim.sgd.SGD -> paddle.optimizer.sgd.SGD（旧SGD）
paddle.optim.SGD 新SGD
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
| weight_decay: float | Tensor = 0, | ||
| nesterov: bool = False, | ||
| ) -> None: | ||
| warnings.warn( |
There was a problem hiding this comment.
传入了再warning吧，不要直接打warning。误以为这是个问题API
| foreach: bool | None = None, | ||
| ) -> None: | ||
| warnings.warn( | ||
| "lr_decay, foreach are currently not supported in Adagrad and will be ignored. " |
There was a problem hiding this comment.
传入了再warning吧，不要直接打warning。误以为这是个问题API
lr_decay上次讨论的是比较容易实现？
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
zhwesky2010
left a comment
zhwesky2010
left a comment
There was a problem hiding this comment.
看下上次讨论的工作完成了没，是否遇到什么困难
| grad_clip: GradientClipBase | None = None, | ||
| name: str | None = None, | ||
| initial_accumulator_value: float = 0.0, | ||
| lr_decay: float = 0.0, |
There was a problem hiding this comment.
这里有一个简单的实现，lr_decay可以通过LRScheduler来实现，这个会封装好global_step这些变量。
另外上次讨论的一些容易实现的参数，实现了没？maximize你在取lr时再取下反。default实现了吗
There was a problem hiding this comment.
maximize目前是准备在基类的step方法中取梯度的时候给梯度取反
defaults准备等maximize实现之后再支持
There was a problem hiding this comment.
这个lr_decay公式代表的应该是InverseTimeDecay，我直接作为成员进行初始化，然后在需要的时候get_lr()应该就可以
CI报告基于以下代码生成（30分钟更新一次）: 1 Required任务 : 43/48 通过
2 失败详情🔴 Static-Check / Test — PR问题（置信度: 高）错误类型: PR问题 | 置信度: 高
关键日志:
修复建议:
关联变更: 🔴 Model-Benchmark / Benchmark test — 环境问题（置信度: 高）错误类型: 环境问题 | 置信度: 高
关键日志:
修复建议:
关联变更: 未发现与 PR 修改文件有关联 🔴 Coverage test — 环境问题（置信度: 高）错误类型: 环境问题 | 置信度: 高
关键日志:
修复建议:
关联变更: 未发现与 PR 修改文件有关联 🟡 Slice / Slice test — 不稳定问题（置信度: 中）错误类型: 不稳定问题 | 置信度: 中
关键日志:
修复建议:
关联变更: 未发现与 PR 修改文件有关联 🟡 Fleet Unit test (single card) — 不稳定问题（置信度: 中）错误类型: 不稳定问题 | 置信度: 中
关键日志:
修复建议:
关联变更: 未发现与 PR 修改文件有关联 |
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
|
添加maximize参数涉及到多个文件修改，下个PR和defaults一并处理 |
PaddlePaddle-bot
left a comment
PaddlePaddle-bot
left a comment
There was a problem hiding this comment.
🤖 Paddle-CI-Agent | pr_review |
2026-06-15 16:28:02
📋 Review 摘要
PR 概述：为 paddle.optim 新增 SGD、Adagrad、AdamW 兼容 wrapper，并给 paddle.optimizer.Adagrad 增加 lr_decay。
变更范围：python/paddle/optim、python/paddle/optimizer/adagrad.py、相关 legacy 测试
影响面 Tag：[User Experience]
问题
| 级别 | 文件 | 概述 |
|---|---|---|
| 🔴 Bug | python/paddle/optimizer/adagrad.py:173 |
lr_decay 的内部 scheduler 状态未进入 optimizer state_dict，断点恢复后衰减步数重置 |
| 🟡 建议 | test/legacy_test/test_adagrad_op.py:462 |
新增 param group 用例没有断言，无法覆盖 lr_decay 组间行为和保存恢复 |
历史 Findings 修复情况
| Finding | 问题 | 状态 |
|---|---|---|
| F1 | paddle.optim.SGD wrapper 仍未兼容原 alias 的 learning_rate/parameters/grad_clip 等参数 |
|
| F2 | paddle.optim.Adagrad wrapper 仍未兼容原 alias 的 learning_rate/parameters/grad_clip/epsilon 等参数 |
|
| F3 | paddle.optim.AdamW wrapper 仍未兼容原 alias 的多项 Paddle 参数 |
|
| F4 | Adagrad.lr_decay 不再完全被忽略，wrapper 已向底层 optimizer 传递该参数 |
✅ 已修复 |
| F5 | SGD 的 momentum/nesterov 仍只是 warning 后忽略，实际仍执行普通 SGD |
|
| F6 | Adagrad.lr_decay 的 step 顺序和静态图更新语义仍未修复 |
|
| F7 | 参数组 lr_decay 仍依赖 optimizer 级 _current_lr_decay_scheduler，静态图/组间状态仍不可靠 |
📝 PR 规范检查
标题 Tag [API Compatibility] 不在 Paddle PR Category/Types 枚举中；描述结构和精度字段齐全。
标题建议（可直接复制）：
[User Experience] Add API aliases for SGD, Adagrad, and AdamW
PR 描述建议（点击展开，可直接复制）
### PR Category
<!-- One of [ User Experience | Execute Infrastructure | Operator Mechanism | CINN | Custom Device | Performance Optimization | Distributed Strategy | Parameter Server | Communication Library | Auto Parallel | Inference | Environment Adaptation ] -->
User Experience
### PR Types
<!-- One of [ New features | Bug fixes | Improvements | Performance | BC Breaking | Deprecations | Docs | Devs | Not User Facing | Security | Others ] -->
Improvements
### Description
<!-- Describe what you’ve done -->
- Add `paddle.optim.SGD`, `paddle.optim.Adagrad`, and `paddle.optim.AdamW` compatibility wrappers with `params`/`lr` style arguments.
- Add API compatibility tests for positional, keyword, and mixed argument construction.
- Remove these optimizers from strict alias identity tests because `paddle.optim` now wraps `paddle.optimizer` implementations.
### 是否引起精度变化
<!-- one of the following [ 是 | 否 ]-->
否总体评价
本轮覆盖了 8 个变更文件中的公开 API wrapper、Adagrad.lr_decay 调用链和新增测试。当前仍有多个历史兼容性/lr_decay 语义问题未修复，且新增 checkpoint 状态问题会影响训练恢复，建议先修正后再合入。
| self._master_weights = {} | ||
| self.initial_accumulator_value = initial_accumulator_value | ||
| self._lr_decay = lr_decay | ||
| self._lr_decay_schedulers = { |
There was a problem hiding this comment.
🔴 Bug lr_decay 的步数状态没有进入 optimizer checkpoint，断点恢复后学习率衰减会从头开始。
这里把 lr_decay 包成私有的 _lr_decay_schedulers，但基类 Optimizer.state_dict() 只会在 _learning_rate 本身是 LRScheduler 时保存 LR_Scheduler。本实现里 _learning_rate 仍是 float，训练若在若干步后保存 optimizer.state_dict() 再恢复，InverseTimeDecay 的 last_epoch 会重新从 0 初始化，后续 Adagrad 更新使用错误的有效学习率。
建议修复方式：将 lr_decay 的 step 状态纳入 Adagrad.state_dict()/set_state_dict()，或改成由可持久化 step accumulator 计算衰减；同时补一个保存、加载 optimizer 后继续训练时有效 lr 与未中断训练一致的回归测试。
| lr_decay=0.0, | ||
| ) | ||
|
|
||
| for epoch in range(3): |
There was a problem hiding this comment.
🟡 建议 这个 param group 用例没有任何断言，无法防住 lr_decay 组间状态错误。
当前测试只运行三步训练，即使第二组的 lr_decay 被忽略、错用默认 scheduler、或保存恢复后重置，也会通过。这里至少应比较两个 group 的有效更新量，或者用可解析的一维参数和固定梯度断言第二组按 lr_decay=0.2 产生的参数值；再增加 state_dict 恢复场景，避免这类 scheduler 状态问题继续漏测。
| else None | ||
| ) | ||
|
|
||
| param_lr = self._create_param_lr(param_and_grad) |
There was a problem hiding this comment.
意思是这里不额外处理lr，直接传入LRScheduler到基类，基类自己会处理LRScheduler。
这个lr_decay对应的是MultiStepDecay？
| 'initial_accumulator_value', | ||
| self._default_dict['initial_accumulator_value'], | ||
| ) | ||
| self._lr_decay = parameters.get( |
There was a problem hiding this comment.
不加额外处理，直接遵循基类的LRScheduler处理
| ] | ||
| return parameters | ||
|
|
||
| def step( |
|
直接把 |
|
预计和其它修改合并提交，本PR关闭 |
4 participants
PR Category
User Experience
PR Types
Improvements
Description
Used AI Studio
是否引起精度变化
否