grammar-check discrepancies from nested block comments

[mattheww]

After #2330, I'm seeing grammar-check discrepancies for input where there's something looking like a doc-comment nested inside a non-doc comment, or vice versa, and the inner one contains a CR.

For example the following cases:

/* /**␍*/ */
/* /*!␍*/ */
/* /** x␍ y */ */
/** /*␍*/ */
/*! /*␍*/ */
/** /* x␍ y */ */

(where ␍ indicates a CR control character)

Running grammar-check:

cargo run -q -p grammar-check -r -- \
lex-compare --tool rustc_parse --edition 2024 \
  --string $'/* /**\r*/ */' \
  --string $'/* /*!\r*/ */' \
  --string $'/* /** x\r y */ */' \
  --string $'/** /*\r*/ */' \
  --string $'/*! /*\r*/ */' \
  --string $'/** /* x\r y */ */' \
2>&1 | sed 's/\r/␍/g'

Output:

------------------------------------------------------------
error: reference failed, rustc_parse passed
test: CLI string
reference error: expected   (~[`*` CR] | BLOCK_COMMENT_OR_DOC)   (BLOCK_COMMENT_OR_DOC | BLOCK_CHAR)*   `*/` at ` x␍ y */ */`
1: /* /** x␍ y */ */
         ━

------------------------------------------------------------
error: reference failed, rustc_parse passed
test: CLI string
reference error: expected (BLOCK_COMMENT_OR_DOC | BLOCK_CHAR)* `*/` at `␍*/ */`
1: /* /*!␍*/ */
         ━

------------------------------------------------------------
error: reference failed, rustc_parse passed
test: CLI string
reference error: expected   (~[`*` CR] | BLOCK_COMMENT_OR_DOC)   (BLOCK_COMMENT_OR_DOC | BLOCK_CHAR)*   `*/` at `␍*/ */`
1: /* /**␍*/ */
         ━

------------------------------------------------------------
error: rustc_parse failed, reference passed
test: CLI string
rustc_parse error: error: <internal>:1:7: error: bare CR not allowed in block doc-comment
 at `␍*/ */`
1: /*! /*␍*/ */
         ━

------------------------------------------------------------
error: rustc_parse failed, reference passed
test: CLI string
rustc_parse error: error: <internal>:1:7: error: bare CR not allowed in block doc-comment
 at `␍*/ */`
1: /** /*␍*/ */
         ━

------------------------------------------------------------
error: rustc_parse failed, reference passed
test: CLI string
rustc_parse error: error: <internal>:1:9: error: bare CR not allowed in block doc-comment
 at `␍ y */ */`
1: /** /* x␍ y */ */
           ━

------------------------------------------------------------
passed: 0
failed: 6
finished in 0.0 seconds

NB the grammar-check output can be hard to read in a terminal because it emits literal CRs. I've replaced them with ␍ above.