test(agent): regression coverage for #2169 skill load tool on fresh sessions#2369
Open
gxgeek-n wants to merge 1 commit into
Open
test(agent): regression coverage for #2169 skill load tool on fresh sessions#2369gxgeek-n wants to merge 1 commit into
gxgeek-n wants to merge 1 commit into
Conversation
|
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds end-to-end regression coverage for #2169 by exercising a full ReAct loop with a real SkillBox + ReActAgent, asserting that load_skill_through_path remains authorized on fresh sessions and across repeated calls.
Changes:
- Introduces
SkillLoadToolFreshSessionTestwith scripted-model e2e tests to ensure the skill load tool executes (notUnauthorized) on the first call of a fresh session. - Verifies
ToolContextStateretains the build-time activeskill-build-in-toolsgroup after the call. - Verifies authorization persists across a second call within the same session.
Comment on lines
+91
to
+96
| // Registered skill ids carry a "_custom" source suffix (see RegisteredSkill); | ||
| // the load tool's skillId enum only accepts the registered form. | ||
| String registeredSkillId = SKILL_ID + "_custom"; | ||
| Map<String, Object> input = new HashMap<>(); | ||
| input.put("skillId", registeredSkillId); | ||
| input.put("path", "SKILL.md"); |
…ol on fresh sessions Add end-to-end tests pinning the fix for agentscope-java#2169: a fresh-session ReActAgent built with a SkillBox must be able to execute load_skill_through_path instead of receiving "Unauthorized tool call: 'load_skill_through_path' is not available". The tests drive a scripted model through a real ReAct loop with a registered skill and assert on the tool result exactly as the model observes it: - fresh session: the load tool executes and returns the skill content - fresh session: ToolContextState keeps skill-build-in-tools activated - second call in the same session: the load tool stays authorized Verified both directions: all three tests fail on the v2.0.0 tag with the exact "Unauthorized tool call" error from the issue, and pass on main with the fresh-slot group seeding from agentscope-ai#2319.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
gxgeek-n
force-pushed
the
test/2169-skill-load-fresh-session
branch
from
July 23, 2026 12:27
3ceaf28 to
0227ef5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds end-to-end regression tests pinning the fix for #2169 (
Unauthorized tool call: 'load_skill_through_path' is not availableon fresh sessions).The bug: on the first call of a fresh session,
activateSlotForContextapplied the state's (empty) activated group list viasetActiveGroups, deactivating the build-time activeskill-build-in-toolsgroup.SkillHookkept advertising skills in the system prompt, so the model inevitably calledload_skill_through_pathand got the Unauthorized error. This was fixed on main by #2319 (fresh slots inherit build-time active groups), but the end-to-end path through a realSkillBox+ReActAgentcall had no coverage — existing tests only assert state seeding with manually-created groups, andSkillBoxTest/SkillBoxToolsTestnever drive a full agent call.What the tests do
SkillLoadToolFreshSessionTestdrives a scripted model through a real ReAct loop with a registered skill, asserting on the tool result exactly as the model observes it:freshSession_skillLoadToolExecutes— the load tool executes and returns the skill content (not Unauthorized)freshSession_stateSeedsSkillGroup—ToolContextStatekeepsskill-build-in-toolsactivated after the callsecondCall_skillLoadToolStillAuthorized— activation survives repeated calls in one session (the wipe happened per call, not just on the first)Verified both directions
Error: Unauthorized tool call: 'load_skill_through_path' is not availablefrom the issue reportNotes
PreCallEvent; happy to drop it once a release ships with fix: preserve skill isolation and tool result history #2319