Skip to content

fix(datasets): apply validator to cached text before serving it - #80

Merged
abhinav-pola merged 1 commit into
mainfrom
devin/1788975547-validate-cached-text
Sep 9, 2026
Merged

fix(datasets): apply validator to cached text before serving it#80
abhinav-pola merged 1 commit into
mainfrom
devin/1788975547-validate-cached-text

Conversation

@devin-ai-integration

Copy link
Copy Markdown
Contributor

TL;DR

fetchCachedTextFile now runs request.validate on cache hits too, so a malformed cached envelope is refetched and overwritten instead of poisoning every run.

What changed?

  • src/datasets/cached-file.ts: cache hit is only returned when request.validate?.(cached.text) === undefined; otherwise fall through to download → validate → write (existing path).
  • src/datasets/cached-file.test.ts: regression test seeding an HTML body under the cache key and asserting one origin request plus the envelope being replaced.

Why?

Follow-up to #79, which added validation before cache writes. Entries written before that (or by any other writer) still bypassed the validator on read, so a pinned cache with an HTML/rate-limit body would fail every benchmark run without ever refetching. Flagged by Devin Review on openrouter-web#41251.

How to test

bun test src/datasets/cached-file.test.ts

Expected: 19 pass, including "refetches and overwrites a cached entry the validator rejects".

Reviewer focus

  • Requests without validate are unchanged (undefined?.(…) === undefined).

Checklist

  • Tests cover changed behavior
  • Public API or configuration changes are backward compatible, or the break is documented
  • Benchmark changes document dataset provenance and licensing
  • No credentials, private results, or restricted dataset contents are included
  • Documentation is updated where needed

Link to Devin session: https://openrouter.devinenterprise.com/sessions/949ffdf48c7843febc66e0c2d8dc8486
Open in Devin Desktop: https://openrouter.devinenterprise.com/desktop/session/949ffdf48c7843febc66e0c2d8dc8486?variant=devin
Requested by: @abhinav-pola

Co-Authored-By: Abhinav Pola <abhinav.pola@openrouter.ai>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR that start with 'DevinAI' or '@devin'.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Original prompt from Abhinav

SYSTEM:
<latest_message>
Abhinav Pola (U090K0G7JF3) [ts=1788974411.868429]: @Devinsync benchmark harness subtree
</latest_message>

=== BEGIN THREAD HISTORY (in #ecosystem-agents) ===
Abhinav Pola (U090K0G7JF3) [ts=1788974411.868429]: @Devinsync benchmark harness subtree
=== END THREAD HISTORY ===
Channel ID: C0BU53A7VEH
Thread URL: https://openrouter.slack.com/archives/C0BU53A7VEH/p1788974411868429?thread_ts=1788974411.868429&amp;cid=C0BU53A7VEH

The <latest_message> is the message that you should use to guide your goals + task for this session, and you should use the rest of the slack thread as context.
A [ts=...] marker on a Slack message is that message's timestamp. To act on a specific message with the slack tool (e.g. adding an emoji reaction via the reaction command), pass that value as timestamp along with the Channel ID — no extra lookup call is needed.

@abhinav-pola
abhinav-pola merged commit 1f92e14 into main Sep 9, 2026
4 checks passed
@abhinav-pola
abhinav-pola deleted the devin/1788975547-validate-cached-text branch September 9, 2026 17:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant