Trustabl Agent scanner - #394
Conversation
📝 WalkthroughWalkthroughThis PR adds a new GitHub Actions workflow file, ChangesTrustabl Workflow Addition
Estimated code review effort: 1 (Trivial) | ~3 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment Warning |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
.github/workflows/trustabl.yml (1)
16-16: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick winDisable checkout credential persistence.
Add
persist-credentials: false.actions/checkout@v4otherwise persists the token for later steps. Trustabl'sgithub-tokeninput already defaults togithub.token, so SARIF uploads and pull-request comments retain their required authentication. (github.com)Suggested fix
- uses: actions/checkout@v4 + with: + persist-credentials: falseAs indicated by static analysis, this addresses the zizmor
artipackedwarning.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In @.github/workflows/trustabl.yml at line 16, Update the actions/checkout@v4 step in the workflow to set persist-credentials to false, while leaving Trustabl’s existing github-token authentication behavior unchanged.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/trustabl.yml:
- Around line 16-17: Update the workflow steps using actions/checkout@v4 and
trustabl/trustabl-action@v0 to reference immutable, full-length commit SHAs
instead of mutable tags, preserving the existing actions and permissions.
---
Nitpick comments:
In @.github/workflows/trustabl.yml:
- Line 16: Update the actions/checkout@v4 step in the workflow to set
persist-credentials to false, while leaving Trustabl’s existing github-token
authentication behavior unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 39b5e898-bb9c-4e6e-8df1-3fa4a8be5249
📒 Files selected for processing (1)
.github/workflows/trustabl.yml
| - uses: actions/checkout@v4 | ||
| - uses: trustabl/trustabl-action@v0 |
There was a problem hiding this comment.
🔒 Security & Privacy | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "Repository files matching trustabl workflow:"
fd -a 'trustabl\.yml$|trustabl\.ya?ml$' .github || true
echo
echo "Trusting workflow content:"
if [ -f .github/workflows/trustabl.yml ]; then
cat -n .github/workflows/trustabl.yml
else
echo ".github/workflows/trustabl.yml not found"
fi
echo
echo "Search for trustabl inputs / permissions / checkout usage:"
rg -n --hidden --glob '!node_modules' --glob '!dist' --glob '!build' \
"trustabl/trustabl-action|version:|permissions:|scan|checkout@v4|uses:.*trustabl|trusted_publishing|oidc|id-token" .github workflows . 2>/dev/null | head -200
echo
echo "Workflow parser safety snapshot:"
python3 - <<'PY'
from pathlib import Path
p=Path('.github/workflows/trustabl.yml')
print(p.exists())
if p.exists():
text=p.read_text()
print(len(text.splitlines()))
for i,line in enumerate(text.splitlines(),1):
if 'uses:' in line or 'with:' in line or 'permissions:' in line or 'id-token' in line or 'version:' in line:
print(f'{i}: {line}')
PYRepository: ElderEvil/falloutProject
Length of output: 19631
🌐 Web query:
trustabltrustabl-action v0 with version latest input documentation
💡 Result:
Trustabl-action is a GitHub Action designed to run the Trustabl static reliability and safety analyzer for AI agent SDKs (such as Claude, OpenAI, Google ADK, and LangChain) [1][2]. It is the recommended, primary method for integrating Trustabl scans into GitHub Actions workflows [3][4][5]. Regarding the versioning: While some documentation references v0 [2], the current authoritative usage examples in active repositories frequently point to @v1 [6][7]. It is recommended to check the official repository for the most stable tag or branch, as the action is designed to be the single source of truth for GitHub-based CI workflows [3][4]. Key features and documentation details for the action include: - SARIF Integration: It natively supports uploading results to the GitHub Security tab, defaulting to true [3][2]. - Configuration: It accepts various inputs for customization, including sarif-file, json-file, upload-artifact, and github-token [2]. - Workflow: For environments outside GitHub Actions, the underlying Trustabl CLI is used with --format sarif --output to produce reports for other tools [3][5]. For the most up-to-date configuration, refer directly to the trustabl/trustabl-action GitHub repository.
Citations:
- 1: https://github.com/trustabl
- 2: https://claudeers.com/trustabl-action
- 3: https://github.com/trustabl/trustabl
- 4: trustabl/trustabl@7310283
- 5: https://github.com/trustabl/trustabl/blob/main/README.md
- 6: Security Findings Detected by Trustabl Scanner AgentOps-AI/agentops#1426
- 7: Noticed some unusual patterns in Trustabl scan Jonhvmp/leopold#42
🌐 Web query:
actions/checkout releases v4 latest
💡 Result:
As of July 31, 2026, the latest release for the v4 major branch of actions/checkout is v4.4.0, which was published on July 20, 2026 [1]. You can reference this specific release in your GitHub Actions workflow using the tag v4.4.0 (e.g., uses: actions/checkout@v4.4.0) [1]. If you prefer to stay on the latest v4 minor/patch updates automatically, you can use the v4 branch reference (e.g., uses: actions/checkout@v4), which is maintained to point to the latest v4 release [2][3]. Note that the actions/checkout repository has also released major versions beyond v4 (up to v7 as of this date) [4][5].
Citations:
- 1: https://github.com/actions/checkout/releases/tag/v4.4.0
- 2: https://github.com/actions/checkout/blob/v4/CHANGELOG.md
- 3: https://github.com/actions/checkout/blob/v4/README.md
- 4: https://github.com/actions/checkout/releases
- 5: https://github.com/actions/checkout
Pin the workflow actions to full-length commit SHAs.
security-events: write and pull-requests: write grant code- and artifact-writing permissions to actions/checkout@v4 and trustabl/trustabl-action@v0. These are mutable tag references, so each job can change to unreviewed code without a workflow diff. Use pinned full-length commit SHAs for both uses entries, or a non-moving released tag only if both repositories support it.
🧰 Tools
🪛 zizmor (1.28.0)
[warning] 16-16: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/trustabl.yml around lines 16 - 17, Update the workflow
steps using actions/checkout@v4 and trustabl/trustabl-action@v0 to reference
immutable, full-length commit SHAs instead of mutable tags, preserving the
existing actions and permissions.
1 participant
We came across your repo and we like the ambitious scope of building a complex simulation game with modern tooling. We scanned the repo, and noticed agent runtime reliability findings that might be worth reviewing.
File: backend/app/services/open_ai.py
What it means: This agent does not constrain its output to a validated type: output_type is absent (defaulting to free-form str) or set explicitly to str.
Recommendations are based on our understanding of agent runtime reliability, some findings may be intentional. Please let us know if this was intentional or if our findings are helpful so we can improve the accuracy of the scanner.
Best,
Trustabl.ai
Open-source AI agent reliability scanner (runs locally, GitHub Action)
Summary by CodeRabbit