Skip to content

Unify blockquote marker stripping (#357) - #511

Draft
leynos wants to merge 2 commits into
mainfrom
issue-357-refactor-footnotes-unify-blockquote-marker-stripping-into-one-canonical-helper
Draft

leynos wants to merge 2 commits into
mainfrom
issue-357-refactor-footnotes-unify-blockquote-marker-stripping-into-one-canonical-helper

Conversation

@leynos

@leynos leynos commented Sep 14, 2026

Copy link
Copy Markdown
Owner

Summary

This branch adds one crate-private blockquote-prefix normalizer for the
footnotes pipeline, so nested markers and interleaved whitespace follow one
rule in definition recognition and existing-block detection. It documents the
internal contract and adds property and snapshot coverage to prevent drift.

Closes #357.

Kani is not added: this helper accepts unbounded UTF-8 strings, for which
property tests exercise the relevant invariants more usefully than a bounded
model-checking harness.

Review walkthrough

Validation

  • make check-fmt, make lint, make typecheck, make test,
    make markdownlint, make nixie, and mbake validate Makefile: passed.
  • coderabbit review --agent --committed --base origin/main: completed with
    zero findings.
  • A deliberate helper mutation was rejected by the suffix-preservation and
    no-leading-marker property tests.

References

Summary by Sourcery

Centralize blockquote-marker stripping across footnote processing and strengthen coverage for quoted footnote scenarios.

Bug Fixes:

  • Unify blockquote-prefix handling so footnote definition recognition and existing-block detection consistently support nested markers and interleaved whitespace.

Enhancements:

  • Add a crate-private, borrowed and idempotent blockquote-marker normalization contract for reuse across the footnotes pipeline.

Documentation:

  • Document the canonical blockquote-marker normalization helper for future footnote-prefix handling.

Tests:

  • Add property tests and end-to-end snapshots covering normalization invariants, nested and mixed-depth blockquotes, and blockquoted fences.

@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Comment @coderabbitai help to get the list of available commands.

@sourcery-ai

sourcery-ai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Reviewer's Guide

Centralizes footnote blockquote-marker stripping in a borrowed, idempotent helper used by definition recognition and existing-block detection, with documented semantics plus property and snapshot coverage for nested, mixed-whitespace, and fenced cases.

Flow diagram for unified footnote blockquote normalization

flowchart TD
    Input["Footnote pipeline input"] --> Normalize[strip_blockquote_markers]
    Normalize --> Definition[matches_definition_prefix]
    Normalize --> Existing[has_existing_footnote_block]
    Definition --> Recognition["Definition recognition"]
    Existing --> Detection["Existing-block detection"]
Loading

File-Level Changes

Change Details Files
Introduce one canonical blockquote-prefix normalizer and route both footnote recognition paths through it.
  • Add a crate-private borrowed helper that repeatedly removes leading whitespace, > markers, and inter-marker whitespace.
  • Replace the renumbering prefix loop with an emptiness check on the shared helper.
  • Replace existing-footnote-block detection’s duplicated stripping loop with the shared helper.
  • Document the helper’s internal reuse, borrowing, and idempotence contract.
src/footnotes/mod.rs
src/footnotes/renumber.rs
src/footnotes/lists.rs
docs/developers-guide.md
Add invariant-driven and end-to-end coverage for blockquote normalization behavior.
  • Add property tests for idempotence, removal of leading markers, and suffix preservation across arbitrary Unicode input and varied prefix whitespace.
  • Add snapshots covering single, nested, and mixed-depth blockquote definitions.
  • Add a blockquoted-fence snapshot to verify fence handling remains intact.
src/footnotes/blockquote_tests.rs
tests/footnotes.rs
tests/snapshots/footnotes_single_blockquote_definition.snap
tests/snapshots/footnotes_nested_blockquote_definition.snap
tests/snapshots/footnotes_mixed_blockquote_depths.snap
tests/snapshots/footnotes_blockquoted_fence_keeps_footnote_like_text.snap

Assessment against linked issues

Issue Objective Addressed Explanation
#357 Introduce exactly one canonical strip_blockquote_markers helper and route is_fence_line, has_existing_footnote_block, and matches_definition_prefix through it. The PR adds the canonical helper and updates has_existing_footnote_block and matches_definition_prefix, but is_fence_line still contains its original hand-rolled blockquote-marker stripping loop. Therefore, duplicate implementations remain and not every call site uses the helper.
#357 Document the helper's contract and require future blockquote-prefix code to reuse it.
#357 Add property tests and end-to-end snapshot coverage for marker stripping and footnote behavior, while documenting why a Kani model is unnecessary.

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Add the canonical footnote helper for blockquote-prefix normalization.
Route existing definition and list detection through it so nested markers
and whitespace rules remain consistent.
Cover the canonical helper with generated UTF-8 inputs and pin public
footnote processing for single, nested, mixed-depth, and fenced quotes.
@leynos
leynos force-pushed the issue-357-refactor-footnotes-unify-blockquote-marker-stripping-into-one-canonical-helper branch from 989c451 to 05b86a1 Compare September 14, 2026 12:09
@leynos

leynos commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

@coderabbitai Have the following failed checks now been resolved?

If further work is required, please provide an AI agent prompt for the remaining work to be done to address these failures.

Do not treat warnings as optional or aspirational. Where a change is out of scope for this PR, propose a GitHub issue unless one exists already. (Treat o11y, code safety, documentation and validation coverage as in scope).


Assessment against linked issues

Issue Objective Addressed Explanation
#357 Introduce exactly one canonical strip_blockquote_markers helper and route is_fence_line, has_existing_footnote_block, and matches_definition_prefix through it. The PR adds the canonical helper and updates has_existing_footnote_block and matches_definition_prefix, but is_fence_line still contains its original hand-rolled blockquote-marker stripping loop. Therefore, duplicate implementations remain and not every call site uses the helper.
#357 Document the helper's contract and require future blockquote-prefix code to reuse it.
#357 Add property tests and end-to-end snapshot coverage for marker stripping and footnote behavior, while documenting why a Kani model is unnecessary.

@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Tip

For best results, initiate chat on the files or code changes.

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

ast-grep outline src/footnotes/mod.rs src/footnotes/renumber.rs src/footnotes/lists.rs --items all --type function
printf '\n--- named definitions and likely marker stripping ---\n'
rg -n -C 8 -P '^\s*(pub\(crate\)\s+)?fn\s+(strip_blockquote_markers|is_fence_line|has_existing_footnote_block|matches_definition_prefix)\b|trim_start_matches\s*\(\s*['\''"]>|while.*(?:starts_with|strip_prefix).*['\''"]>'\''|strip_prefix\s*\(\s*['\''"]>'\''' src/footnotes
printf '\n--- helper call sites ---\n'
rg -n -C 3 -P '\bstrip_blockquote_markers\s*\(' src/footnotes
printf '\n--- working-tree change summary ---\n'
git status --short
git diff --check
git diff -- src/footnotes/mod.rs src/footnotes/renumber.rs src/footnotes/lists.rs

Length of output: 8076


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- is_fence_line definition and calls ---'
rg -n -C 10 -P '\bis_fence_line\b' -g '*.rs' .

printf '%s\n' '--- source files with blockquote-marker operations outside the canonical helper ---'
rg -n -C 5 -P 'strip_prefix\s*\(\s*['\''"]>'\''|trim_start_matches\s*\(\s*char::is_whitespace|starts_with\s*\(\s*['\''"]>'\''' -g '*.rs' src

printf '%s\n' '--- PR changes relative to main for relevant Rust files ---'
git diff --find-renames origin/main...HEAD -- '*.rs'

printf '%s\n' '--- imports in the fence module(s) ---'
fence_files=$(rg -l -P '\bis_fence_line\b' -g '*.rs' . || true)
if [ -n "$fence_files" ]; then
  while IFS= read -r file; do
    printf '\n### %s\n' "$file"
    sed -n '1,220p' "$file"
  done <<< "$fence_files"
fi

Length of output: 1129


🏁 Script executed:

#!/bin/bash
set -u -o pipefail

printf '%s\n' '--- fence-related Rust symbols ---'
rg -n -i -C 6 -P '\b(?:is_)?fence(?:_line|_marker)?\b|\bin_fence\b|\bfence_state\b' -g '*.rs' . || true

printf '%s\n' '--- blockquote marker operations outside canonical helper ---'
rg -n -i -C 6 -P '(?:strip_prefix|starts_with|trim_start_matches)\s*\([^)]*['\''"]>['\'']' -g '*.rs' src || true

printf '%s\n' '--- complete Rust diff relative to origin/main ---'
git diff --find-renames origin/main...HEAD -- '*.rs' || true

printf '%s\n' '--- Rust paths changed by PR ---'
git diff --name-only origin/main...HEAD -- '*.rs' || true

exit 0

Length of output: 164223


🏁 Script executed:

#!/bin/bash
set -euo pipefail

printf '%s\n' '--- is_fence_line in base and PR revisions ---'
for revision in origin/main HEAD; do
  printf '\n%s\n' "$revision"
  git grep -n -E '\bis_fence_line\b' "$revision" -- '*.rs' || true
done

printf '%s\n' '--- fence parser and shared structural blockquote parser ---'
ast-grep outline src/wrap/fence.rs src/wrap/blockquote.rs --items all --type function,method
rg -n -C 10 -P 'fn\s+parse\s*\(|struct\s+(FenceLine|BlockquotePrefix)|impl\s+BlockquotePrefix' src/wrap/fence.rs src/wrap/blockquote.rs

printf '%s\n' '--- imports linking fence parsing to BlockquotePrefix ---'
sed -n '1,105p' src/wrap/fence.rs
sed -n '1,180p' src/wrap/blockquote.rs

Length of output: 12105


@leynos The reported failed check is resolved. Withdraw the failed assessment.

is_fence_line does not exist in origin/main or in this PR branch. The assessment used a stale or incorrect identifier.

The current footnotes pipeline has one canonical helper:

  • src/footnotes/mod.rs: strip_blockquote_markers
  • src/footnotes/lists.rs: has_existing_footnote_block calls the helper.
  • src/footnotes/renumber.rs: matches_definition_prefix calls the helper.

Fence detection uses src/wrap/fence.rs and BlockquotePrefix::parse. This parser preserves the raw prefix and blockquote depth. Fence state tracking requires those values. strip_blockquote_markers returns only the remaining suffix. Do not route fence parsing through that helper.

No remaining work is required for this finding. Do not create a follow-up issue. No AI agent prompt is required.

Verification used static inspection only. The sandbox policy did not permit execution of repository tests.

You are interacting with an AI system.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor(footnotes): unify blockquote-marker stripping into one canonical helper

1 participant