ScaffBench 2.1: address Codex feedback on #258/#259#261
Conversation
- #259 (P1) tsc fallback: use `tsc --build` instead of `--noEmit`. `--noEmit` only checks the root project, so a root tsconfig with `files: []` + project references exits 0 even with type errors in apps/packages; `--build` descends into references, closing the dodge. - #258 (P2) acceptance matching: match by exact dependency / scoped-package prefix / file path instead of substring over all project text — so short tokens like `ai` (⊂ tailwindcss) or `vite` (⊂ vitest) no longer inflate the Acceptance column. `ci` set uses `.circleci` as a path. - #258 (P2) no-project discovery runs now score 0 acceptance (emptyAcceptanceScore) instead of undefined, so maybeAverage no longer drops them and overstates the cell's Acceptance. +2 tests (substring no-credit + testing-still-credited; no-project counted as 0 in the average). 33 pass; type-check unchanged.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
github-actions
Bot
added
size:M
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f61355c15c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if (pattern.startsWith(".")) { | ||
| return files.some((file) => file === pattern || file.includes(`${pattern}/`)); | ||
| } | ||
| return deps.some((dep) => dep === pattern || dep.startsWith(`${pattern}/`)); |
There was a problem hiding this comment.
In natural discovery scoring, a project that uses Auth.js via packages such as @auth/core or @auth/drizzle-adapter should satisfy the auth acceptance set because that set includes "@auth/", but this helper appends another slash and tests for @auth//. That valid Auth.js alternative will now be marked as a missed auth capability and lower acceptance scores; normalize trailing slashes or use dep.startsWith(pattern) when the pattern already ends with /.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in #263: a pattern already ending in / (like @auth/) is now used as-is as the scope prefix instead of appending a second slash, so @auth/core/@auth/drizzle-adapter satisfy the auth capability.
…llowup2 ScaffBench 2.1: fix scoped-prefix acceptance matching (Codex #261)
1 participant
Summary
Addresses the three Codex findings that landed as #258/#259 merged on substantive-green.
tsc --noEmitonly checks the root project, so a roottsconfig.jsonwithfiles: []+referencesexits 0 even with type errors inapps/. Switched the fallback totsc --build, which descends into project references.ai(⊂tailwindcss) andvite(⊂vitest) no longer credit the ai / web-framework capabilities.ciuses.circleci.undefined, somaybeAveragecounts it rather than dropping it.Validation
bun test scripts/scaffbench-v2-lib.test.ts— 33 pass (+2: substring no-credit while exact still credits; no-project counted as 0 in the average). Type-check unchanged.