feat: 新增 ScreenshotTargetExpand 截图缩放模式 (面向 Unity 游戏的模板匹配场景)#1336
Open
ZeroAd-06 wants to merge 1 commit into
Open
feat: 新增 ScreenshotTargetExpand 截图缩放模式 (面向 Unity 游戏的模板匹配场景)#1336ZeroAd-06 wants to merge 1 commit into
ZeroAd-06 wants to merge 1 commit into
Conversation
按 Unity Canvas Scaler "Expand" 语义,以参考分辨率 (W, H) 为目标等比缩放
截图: scale = max(W / raw_w, H / raw_h)。输出两边均不小于参考,保持源宽
高比,不裁剪、不非等比拉伸。
动机:很多 Unity 游戏在不同设备宽高比下会扩展画布而非裁剪,如果继续使用
ScreenshotTargetLongSide/ShortSide 只能锁定一个轴,另一个轴会随设备宽高
比漂移,导致按设计分辨率制作的模板在宽屏/窄屏设备上不再对齐。新模式让
两个轴都不低于参考分辨率,模板匹配可以稳定按设计坐标进行,只需在像素稍
多的一侧做 ROI 裁剪即可,无需为每种屏幕比例分别准备模板。
- 选项值: int32_t[2] = { width, height }
- 与 LongSide / ShortSide 互斥,设置任一会重置其他;UseRawSize 仍是最高
优先级开关
- 触控坐标反向缩放沿用既有 raw / target 比例映射,无需额外处理
- 同步更新 Python / NodeJS 绑定、AgentClient/AgentServer IPC、
ProjectInterface JSON 字段 display_expand、interface.schema.json
及中英文文档
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - 我发现了两个问题，并给出了一些整体性的建议：
- 在
ControllerAgent::calc_target_image_size中，expand 分支在没有检查image_raw_width_/image_raw_height_是否大于 0 的情况下就进行了除法；建议参考其他位置已有的校验逻辑，在原始尺寸未初始化时避免出现未定义行为。 - 在
AgentClient::handle_controller_set_option中处理ScreenshotTargetExpand时，目前只检查了is_number()，但随后调用的是as_integer()；如果 JSON 中包含非整数的数值，行为可能会比较意外，更安全的做法是要么显式验证其为整数，要么使用as_number()再进行类型转换。
给 AI Agent 的提示词
Please address the comments from this code review:
## Overall Comments
- In `ControllerAgent::calc_target_image_size`, the expand branch divides by `image_raw_width_` / `image_raw_height_` without checking they’re > 0; consider mirroring the existing validation used elsewhere to avoid undefined behavior when raw size isn’t initialized.
- In `AgentClient::handle_controller_set_option` for `ScreenshotTargetExpand`, you only check `is_number()` but then call `as_integer()`; if the JSON contains non-integer numbers this may behave unexpectedly, so it might be safer to either validate as integer explicitly or use `as_number()` and cast.
## Individual Comments
### Comment 1
<location path="source/MaaFramework/Controller/ControllerAgent.cpp" line_range="1102-1105" />
<code_context>
return true;
}
+ if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
+ double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
+ double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
+ double scale = std::max(sx, sy);
+ image_target_width_ = static_cast<int>(std::round(image_raw_width_ * scale));
+ image_target_height_ = static_cast<int>(std::round(image_raw_height_ * scale));
</code_context>
<issue_to_address>
**suggestion:** 在这里使用 std::round 可能会破坏“尺寸 ≥ 参考尺寸”的保证；在 expand 模式下建议改用 ceil。
当 `scale = max(W/raw_w, H/raw_h)` 时，浮点误差加上 `std::round` 可能导致结果比预期小 1 个像素（例如 `raw_w * scale` 计算结果略低于理想值）。在计算缩放后的宽高时改为使用 `std::ceil`，可以始终保证满足“≥ 参考尺寸”的约定。
建议实现如下：
```cpp
if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
double scale = std::max(sx, sy);
image_target_width_ = static_cast<int>(std::ceil(image_raw_width_ * scale));
image_target_height_ = static_cast<int>(std::ceil(image_raw_height_ * scale));
LogInfo << "expand" << VAR(scale) << VAR(image_target_width_) << VAR(image_target_height_);
return true;
}
```
如果 `source/MaaFramework/Controller/ControllerAgent.cpp` 中尚未包含 `<cmath>`，请在其他标准库头文件附近添加：
`#include <cmath>`
以便使用 `std::ceil`。
</issue_to_address>
### Comment 2
<location path="source/MaaAgentClient/Client/AgentClient.cpp" line_range="2493-2498" />
<code_context>
+ const auto& arr = req.value.as_array();
+ bool ok = true;
+ for (size_t i = 0; i < 2; ++i) {
+ if (!arr[i].is_number()) {
+ LogError << "ScreenshotTargetExpand array element must be a number" << VAR(arr[i].type_name());
+ ok = false;
+ break;
+ }
+ dims[i] = static_cast<int32_t>(arr[i].as_integer());
+ }
+ if (!ok || dims[0] <= 0 || dims[1] <= 0) {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** 当 expand 维度传入非整数数值时，行为可能会让人意外。
`is_number()` 会接受浮点值，但代码后面使用的是 `as_integer()` 并转换为 `int32_t`，这可能会产生静默截断。如果只接受整数尺寸，建议改为更严格的检查（例如使用 `is_integer()`（若可用），或者显式验证数值在 `int32_t` 范围内且没有小数部分），以避免令人意外的行为。
建议实现如下：
```cpp
int32_t dims[2] = { 0, 0 };
const auto& arr = req.value.as_array();
bool ok = true;
for (size_t i = 0; i < 2; ++i) {
if (!arr[i].is_integer()) {
LogError << "ScreenshotTargetExpand array element must be an integer number"
<< VAR(arr[i].type_name());
ok = false;
break;
}
const auto v = arr[i].as_integer();
if (v < std::numeric_limits<int32_t>::min() || v > std::numeric_limits<int32_t>::max()) {
LogError << "ScreenshotTargetExpand array element is out of int32_t range"
<< VAR(v);
ok = false;
break;
}
dims[i] = static_cast<int32_t>(v);
}
if (!ok || dims[0] <= 0 || dims[1] <= 0) {
```
如果当前翻译单元中还未引入 `std::numeric_limits`，请在 `source/MaaAgentClient/Client/AgentClient.cpp` 顶部附近添加 `#include <limits>`。
如果你使用的 JSON/值类型不提供 `is_integer()`，请用等价的检查替代（例如 `is_int64()`，或者结合 `is_number()` 与小数部分检查），并保持与所用 JSON 库 API 一致。
</issue_to_address>帮我变得更有用！请在每条评论上点 👍 或 👎，我会根据你的反馈改进 Review 质量。
Original comment in English
Hey - I've found 2 issues, and left some high level feedback:
- In
ControllerAgent::calc_target_image_size, the expand branch divides byimage_raw_width_/image_raw_height_without checking they’re > 0; consider mirroring the existing validation used elsewhere to avoid undefined behavior when raw size isn’t initialized. - In
AgentClient::handle_controller_set_optionforScreenshotTargetExpand, you only checkis_number()but then callas_integer(); if the JSON contains non-integer numbers this may behave unexpectedly, so it might be safer to either validate as integer explicitly or useas_number()and cast.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `ControllerAgent::calc_target_image_size`, the expand branch divides by `image_raw_width_` / `image_raw_height_` without checking they’re > 0; consider mirroring the existing validation used elsewhere to avoid undefined behavior when raw size isn’t initialized.
- In `AgentClient::handle_controller_set_option` for `ScreenshotTargetExpand`, you only check `is_number()` but then call `as_integer()`; if the JSON contains non-integer numbers this may behave unexpectedly, so it might be safer to either validate as integer explicitly or use `as_number()` and cast.
## Individual Comments
### Comment 1
<location path="source/MaaFramework/Controller/ControllerAgent.cpp" line_range="1102-1105" />
<code_context>
return true;
}
+ if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
+ double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
+ double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
+ double scale = std::max(sx, sy);
+ image_target_width_ = static_cast<int>(std::round(image_raw_width_ * scale));
+ image_target_height_ = static_cast<int>(std::round(image_raw_height_ * scale));
</code_context>
<issue_to_address>
**suggestion:** Using std::round here can violate the ">= reference size" guarantee; consider ceil for expand mode.
With `scale = max(W/raw_w, H/raw_h)`, floating-point error plus `std::round` can yield dimensions 1 pixel smaller than guaranteed (e.g., `raw_w * scale` ends up just below the ideal value). Using `std::ceil` when computing the scaled width/height would consistently enforce the ">= reference" contract.
Suggested implementation:
```cpp
if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
double scale = std::max(sx, sy);
image_target_width_ = static_cast<int>(std::ceil(image_raw_width_ * scale));
image_target_height_ = static_cast<int>(std::ceil(image_raw_height_ * scale));
LogInfo << "expand" << VAR(scale) << VAR(image_target_width_) << VAR(image_target_height_);
return true;
}
```
If `source/MaaFramework/Controller/ControllerAgent.cpp` does not already include `<cmath>`, add:
`#include <cmath>`
near the other standard library includes so that `std::ceil` is available.
</issue_to_address>
### Comment 2
<location path="source/MaaAgentClient/Client/AgentClient.cpp" line_range="2493-2498" />
<code_context>
+ const auto& arr = req.value.as_array();
+ bool ok = true;
+ for (size_t i = 0; i < 2; ++i) {
+ if (!arr[i].is_number()) {
+ LogError << "ScreenshotTargetExpand array element must be a number" << VAR(arr[i].type_name());
+ ok = false;
+ break;
+ }
+ dims[i] = static_cast<int32_t>(arr[i].as_integer());
+ }
+ if (!ok || dims[0] <= 0 || dims[1] <= 0) {
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Potentially surprising behavior when a non-integer number is passed for expand dimensions.
`is_number()` allows floating-point values, but the code then uses `as_integer()` and casts to `int32_t`, which may silently truncate. If only integral dimensions are valid, consider a stricter check (e.g., `is_integer()` if available, or explicitly verifying the value is within `int32_t` range and has no fractional part) to avoid surprising behavior.
Suggested implementation:
```cpp
int32_t dims[2] = { 0, 0 };
const auto& arr = req.value.as_array();
bool ok = true;
for (size_t i = 0; i < 2; ++i) {
if (!arr[i].is_integer()) {
LogError << "ScreenshotTargetExpand array element must be an integer number"
<< VAR(arr[i].type_name());
ok = false;
break;
}
const auto v = arr[i].as_integer();
if (v < std::numeric_limits<int32_t>::min() || v > std::numeric_limits<int32_t>::max()) {
LogError << "ScreenshotTargetExpand array element is out of int32_t range"
<< VAR(v);
ok = false;
break;
}
dims[i] = static_cast<int32_t>(v);
}
if (!ok || dims[0] <= 0 || dims[1] <= 0) {
```
If `std::numeric_limits` is not already available in this translation unit, add `#include <limits>` near the top of `source/MaaAgentClient/Client/AgentClient.cpp`.
If the JSON/value type you're using does not provide `is_integer()`, replace that call with an equivalent (e.g., `is_int64()` or a combination of `is_number()` and a fractional-part check) consistent with your JSON library’s API.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
Comment on lines
+1102
to
+1105
| if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) { | ||
| double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_; | ||
| double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_; | ||
| double scale = std::max(sx, sy); |
Contributor
There was a problem hiding this comment.
suggestion: 在这里使用 std::round 可能会破坏“尺寸 ≥ 参考尺寸”的保证；在 expand 模式下建议改用 ceil。
当 scale = max(W/raw_w, H/raw_h) 时，浮点误差加上 std::round 可能导致结果比预期小 1 个像素（例如 raw_w * scale 计算结果略低于理想值）。在计算缩放后的宽高时改为使用 std::ceil，可以始终保证满足“≥ 参考尺寸”的约定。
建议实现如下：
if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
double scale = std::max(sx, sy);
image_target_width_ = static_cast<int>(std::ceil(image_raw_width_ * scale));
image_target_height_ = static_cast<int>(std::ceil(image_raw_height_ * scale));
LogInfo << "expand" << VAR(scale) << VAR(image_target_width_) << VAR(image_target_height_);
return true;
}
如果 source/MaaFramework/Controller/ControllerAgent.cpp 中尚未包含 <cmath>，请在其他标准库头文件附近添加：
#include <cmath>
以便使用 std::ceil。
Original comment in English
suggestion: Using std::round here can violate the ">= reference size" guarantee; consider ceil for expand mode.
With scale = max(W/raw_w, H/raw_h), floating-point error plus std::round can yield dimensions 1 pixel smaller than guaranteed (e.g., raw_w * scale ends up just below the ideal value). Using std::ceil when computing the scaled width/height would consistently enforce the ">= reference" contract.
Suggested implementation:
if (image_target_expand_width_ > 0 && image_target_expand_height_ > 0) {
double sx = static_cast<double>(image_target_expand_width_) / image_raw_width_;
double sy = static_cast<double>(image_target_expand_height_) / image_raw_height_;
double scale = std::max(sx, sy);
image_target_width_ = static_cast<int>(std::ceil(image_raw_width_ * scale));
image_target_height_ = static_cast<int>(std::ceil(image_raw_height_ * scale));
LogInfo << "expand" << VAR(scale) << VAR(image_target_width_) << VAR(image_target_height_);
return true;
}
If source/MaaFramework/Controller/ControllerAgent.cpp does not already include <cmath>, add:
#include <cmath>
near the other standard library includes so that std::ceil is available.
✅ Addressed in 4f27e6a: The expand branch now uses std::ceil for both dimensions and includes a comment explaining the reasoning, ensuring the scaled size always meets the “>= reference” guarantee as suggested.
Comment on lines
+2493
to
+2498
| if (!arr[i].is_number()) { | ||
| LogError << "ScreenshotTargetExpand array element must be a number" << VAR(arr[i].type_name()); | ||
| ok = false; | ||
| break; | ||
| } | ||
| dims[i] = static_cast<int32_t>(arr[i].as_integer()); |
Contributor
There was a problem hiding this comment.
suggestion (bug_risk): 当 expand 维度传入非整数数值时，行为可能会让人意外。
is_number() 会接受浮点值，但代码后面使用的是 as_integer() 并转换为 int32_t，这可能会产生静默截断。如果只接受整数尺寸，建议改为更严格的检查（例如使用 is_integer()（若可用），或者显式验证数值在 int32_t 范围内且没有小数部分），以避免令人意外的行为。
建议实现如下：
int32_t dims[2] = { 0, 0 };
const auto& arr = req.value.as_array();
bool ok = true;
for (size_t i = 0; i < 2; ++i) {
if (!arr[i].is_integer()) {
LogError << "ScreenshotTargetExpand array element must be an integer number"
<< VAR(arr[i].type_name());
ok = false;
break;
}
const auto v = arr[i].as_integer();
if (v < std::numeric_limits<int32_t>::min() || v > std::numeric_limits<int32_t>::max()) {
LogError << "ScreenshotTargetExpand array element is out of int32_t range"
<< VAR(v);
ok = false;
break;
}
dims[i] = static_cast<int32_t>(v);
}
if (!ok || dims[0] <= 0 || dims[1] <= 0) {
如果当前翻译单元中还未引入 std::numeric_limits，请在 source/MaaAgentClient/Client/AgentClient.cpp 顶部附近添加 #include <limits>。
如果你使用的 JSON/值类型不提供 is_integer()，请用等价的检查替代（例如 is_int64()，或者结合 is_number() 与小数部分检查），并保持与所用 JSON 库 API 一致。
Original comment in English
suggestion (bug_risk): Potentially surprising behavior when a non-integer number is passed for expand dimensions.
is_number() allows floating-point values, but the code then uses as_integer() and casts to int32_t, which may silently truncate. If only integral dimensions are valid, consider a stricter check (e.g., is_integer() if available, or explicitly verifying the value is within int32_t range and has no fractional part) to avoid surprising behavior.
Suggested implementation:
int32_t dims[2] = { 0, 0 };
const auto& arr = req.value.as_array();
bool ok = true;
for (size_t i = 0; i < 2; ++i) {
if (!arr[i].is_integer()) {
LogError << "ScreenshotTargetExpand array element must be an integer number"
<< VAR(arr[i].type_name());
ok = false;
break;
}
const auto v = arr[i].as_integer();
if (v < std::numeric_limits<int32_t>::min() || v > std::numeric_limits<int32_t>::max()) {
LogError << "ScreenshotTargetExpand array element is out of int32_t range"
<< VAR(v);
ok = false;
break;
}
dims[i] = static_cast<int32_t>(v);
}
if (!ok || dims[0] <= 0 || dims[1] <= 0) {
If std::numeric_limits is not already available in this translation unit, add #include <limits> near the top of source/MaaAgentClient/Client/AgentClient.cpp.
If the JSON/value type you're using does not provide is_integer(), replace that call with an equivalent (e.g., is_int64() or a combination of is_number() and a fractional-part check) consistent with your JSON library’s API.
ZeroAd-06
force-pushed
the
feat/screenshot-target-expand
branch
from
4f27e6a to
e9b1ebb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
新增控制器选项
MaaCtrlOption_ScreenshotTargetExpand,接收参考分辨率(width, height),按 Unity Canvas Scaler Expand 语义对截图等比缩放:保持源宽高比,输出两边均不小于参考;无裁剪、无非等比拉伸。
Motivation
针对采用 Unity Canvas Scaler Expand 模式的游戏做模板匹配,本 PR
解决的是 UI 元素在截图中的像素大小 问题 —— 让缩放后图像里 UI
元素的像素尺寸与设计阶段的模板一致,使模板匹配能正常工作。
Unity Expand 模式下:
设计阶段为参考分辨率
(ref_w, ref_h)制作的e像素 UI 元素,在屏幕上实际渲染为
e * unityScale像素。屏幕宽高比变化时unityScale会随之变化,导致同一 UI 元素在不同设备上的实际像素大小不同。
模板匹配要求模板和截图中元素的像素尺寸一致。现有的 LongSide /
ShortSide 缩放只锁单轴:
对齐,选错则缩放后 UI 元素像素大小 ≠ 模板,匹配失败(不是位置错
位,是模板的像素尺寸与图像中 UI 的像素尺寸对不上)
LongSide / ShortSide,实际部署中很难假设
新模式把 scale 算成:
这正是 Unity scaleFactor 的倒数。无论源屏比例宽于还是窄于参考宽
高比,自适应地选对要锁的轴,缩放后图像里 UI 元素的像素尺寸恒等于
设计模板尺寸。集成方一次配置
(ref_w, ref_h),对所有设备宽高比一致工作。
举例验证
参考
1280×720:最后一行(1920×1280,源比参考更「方」)用 ShortSide=720 会得到
scale=0.5625,缩放后 100px 元素变 84.4px,与模板对不上;LongSide=1280
正确。倒数第二行(竖屏)反过来:ShortSide 对、LongSide 错。Expand
自动选对。
改动范围
MaaCtrlOption_ScreenshotTargetExpand = 8,值为int32_t[2]ControllerAgent: 新增字段与 setter,在calc_target_image_size()内加入 expand 分支;与 LongSide / ShortSide 互斥(设置任一重置其他),
UseRawSize仍是最高优先级开关Controller.set_screenshot_target_expand(width, height)controller.set_screenshot_target_expand(width, height)(因带两参,用
MAA_BIND_FUNC而非 setter)display_expand: [width, height]字段,MaaPiCli在 raw 与 long_side 之间插入新分支tools/interface.schema.json同步Test plan
4:3 / 竖屏),切换四种缩放模式,核对输出尺寸符合公式表
回映到原始坐标
interface.json含\"display_expand\": [1280, 720]时,MaaPiCli 日志中
image_target_*应符合预期兼容性
新选项只是新增;未设置时行为与之前完全一致。LongSide / ShortSide /
UseRawSize 的行为均未改变。