Update CLAUDE.md/AGENTS.md inline context to respect llms.txt#3163
Update CLAUDE.md/AGENTS.md inline context to respect llms.txt#3163
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Use the checkbox below for a quick retry:
WalkthroughDocumentation updates reorder AI-LLM sourcing into an explicit, stepwise hierarchy: always fetch https://ably.com/llms.txt first; treat https://ably.com/docs as the canonical reference; use docs to resolve conflicts; and use web searches only as a fallback after consulting llms.txt. Changes
Estimated code review effort🎯 2 (Simple) | ⏱️ ~8 minutes Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Tip Try Coding Plans. Let us write the prompt for your AI agent so you can ship faster (with fewer bugs). Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
| ## Source of truth: Ably Docs | ||
|
|
||
| For anything related to Ably (APIs, SDKs, limits, behavior, authentication, capabilities, feature availability, deprecations): | ||
| For anything related to Ably: |
There was a problem hiding this comment.
the simplified For anything related to Ably: is much cleaner and all-encompassing, because an exhaustive list (APIs, SDKs etc) might make an LLM think if it's not in this list, maybe these rules don't apply, same behaviour was partially seen in https://github.com/ably-labs/chat-poc-kotlin/pull/12.
There was a problem hiding this comment.
Also, extra tokens don't add any actionable information
| - Use **https://ably.com/llms.txt** as the authoritative index of Ably documentation. | ||
| - Prefer to perform web searches for up to date information from the Ably Docs site. | ||
| - If your prior knowledge, assumptions, or training data conflict with the docs, **the docs always win**. | ||
| 1. Always fetch **https://ably.com/llms.txt** first, this is the primary search index. Use it to identify latest documentation URLs before fetching specific pages. Do not skip this step or jump directly to web searches. |
There was a problem hiding this comment.
search index - creates cognitive link: "use this instead of web search"
latest - addresses the freshness problem—LLMs shouldn't rely on stale training data for URL patterns.
There was a problem hiding this comment.
authoritative implies trust/correctness but doesn't convey "use first", also second point
2. Treat **https://ably.com/docs** as the canonical source of truth already highlights ably.com as trustworthy source of information.
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
GregHolmes
force-pushed
the
fix/building-with-llms-inline-context
branch
from
ca578ee to
669c991
Compare
GregHolmes
force-pushed
the
fix/building-with-llms-inline-context
branch
from
669c991 to
28ca5a6
Compare
|
@coderabbitai review |
✅ Actions performedReview triggered.
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
src/pages/docs/platform/ai-llms/index.mdx (1)
239-239:⚠️ Potential issue | 🟡 MinorTip
#3"Prefer web search" directly contradicts the updated prompt hierarchy.The LLM configuration prompt (lines 53–56) now explicitly demotes web search to a last resort ("may be used as a fallback only after consulting
llms.txt"), but the unchanged user-facing tip still tells developers to prefer web search as the freshness mechanism. A developer reading this page receives two opposing instructions: the prompt they paste intoCLAUDE.mdsays "llms.txt first, web search last," while the tip says the opposite. This undermines the PR's core objective.The tip should be updated to align with the new hierarchy — e.g., framing web search as an optional fallback and llms.txt as the primary index.
📝 Proposed fix
-3. **Prefer web search**: Allow your LLM to perform web searches so it can fetch the most current information from the Ably Docs site, as opposed to relying on potentially outdated training data. +3. **Enable web search as a fallback**: Allow your LLM to perform web searches so it can supplement information not found via `llms.txt` or the Ably Docs pages, rather than relying on potentially outdated training data.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@src/pages/docs/platform/ai-llms/index.mdx` at line 239, Update the Tip `#3` copy titled "Prefer web search" so it matches the new prompt hierarchy: make llms.txt the primary source and present web search as an optional fallback. Concretely, replace the sentence that recommends preferring web search with wording that instructs developers to consult llms.txt first (or embed the Ably Docs index) and only use web search if llms.txt doesn't contain the needed info, and make sure the example prompt references (CLAUDE.md, llms.txt) remain consistent with this hierarchy.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@src/pages/docs/platform/ai-llms/index.mdx`:
- Line 239: Update the Tip `#3` copy titled "Prefer web search" so it matches the
new prompt hierarchy: make llms.txt the primary source and present web search as
an optional fallback. Concretely, replace the sentence that recommends
preferring web search with wording that instructs developers to consult llms.txt
first (or embed the Ably Docs index) and only use web search if llms.txt doesn't
contain the needed info, and make sure the example prompt references (CLAUDE.md,
llms.txt) remain consistent with this hierarchy.
ℹ️ Review info
Configuration used: Repository UI
Review profile: CHILL
Plan: Pro
Disabled knowledge base sources:
- Jira integration is disabled
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
src/pages/docs/platform/ai-llms/index.mdx
sacOO7
force-pushed
the
fix/building-with-llms-inline-context
branch
from
28ca5a6 to
657f597
Compare
…fetch llms.txt first and then fallback to WebSearch
sacOO7
force-pushed
the
fix/building-with-llms-inline-context
branch
from
657f597 to
3a81f87
Compare
3 participants
Summary by CodeRabbit