fix: require word boundaries for mrkdwn emphasis markers - #96
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the mrkdwn inline parser (parseInline) to enforce Slack-like word-boundary rules for emphasis markers, preventing underscores inside URLs/identifiers from pairing with unrelated underscores later in a message and corrupting rendered output.
Changes:
- Add word-boundary checks so a marker cannot open if preceded by a word character, and cannot close if followed by a word character.
- Update closing-marker search to skip mid-word candidates instead of stopping at the first match.
- Add regression tests to ensure underscores in URLs/identifiers are not misinterpreted as emphasis while
_italic_still parses.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| src/lib/mrkdwn.ts | Adds word-boundary logic for opening/closing emphasis spans and improves closing-marker search behavior. |
| src/lib/mrkdwn.test.ts | Adds regression tests for underscore parsing in URLs/identifiers and mixed cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| function isWordChar(ch: string | undefined): boolean { | ||
| return ch !== undefined && /[A-Za-z0-9]/.test(ch); | ||
| } |
There was a problem hiding this comment.
Fixed in a54e3d9: switched to /[\p{L}\p{N}]/u so any Unicode letter or number counts as a word character, not just ASCII.
| it('does not open emphasis on a marker directly after a word character', () => { | ||
| expect(elements('snake_case_name stays literal')).toEqual([ | ||
| { type: 'text', text: 'snake_case_name stays literal' }, | ||
| ]); | ||
| }); |
There was a problem hiding this comment.
Added in a54e3d9: a regression test with cafe_italic_ that fails against the old ASCII-only check and passes with the Unicode-aware one.
There was a problem hiding this comment.
Thanks for tackling this. The underscore-in-URL bug is real and worth fixing, but applying the word-boundary rule uniformly to all four markers introduces regressions. Requesting changes.
Confirmed regressions (traced old vs new output):
**bold**-> old['*', bold('bold'), '*'], new[bold('*bold'), '*']. The opener skips the adjacent*and pairs with the far one, so a literal*lands inside the bold span.~~strike~~-> old['~', strike('strike'), '~'], new[strike('~strike'), '~']. Same mechanism.see https://host/~user and ~important~-> old strikes onlyimportant, new strikesuser and ~important. This is the same swallow bug the PR set out to fix, just moved from_to~.*a*b,*bold*2,a`b`c-> now render fully literal. Per Slack's own docs*bold*and backtick code spans work mid-word; only_italic_requires boundaries. The uniform rule drops valid Slack formatting.
Lower severity:
isWordCharindexes by UTF-16 code unit. NFDcafé_italic_italicizes while NFC stays literal (the added test only covers NFC, masking this); astral letters are misclassified via lone surrogates. UsecodePointAtor normalize to NFC.
Suggested direction: scope the word-boundary rule to _ only (matching Slack), and separately fix the "skip to a later same-marker candidate" logic so it does not merge unrelated spans.
Process: per the repo guidelines, every PR must link an existing issue that carries the ready-for-pr label. This PR has no linked issue and no label, so it cannot be merged as-is regardless of the code.
parseInline paired a marker with the next occurrence of the same character anywhere later in the text, with no check on what came before or after either marker. An underscore inside one identifier or URL (the merge_requests path segment in a GitLab link, say) paired with an unrelated underscore much later in the message, italicising everything between them and deleting both underscores, which destroyed both URLs. An underscore now only opens a span when it is not preceded by a word character, and only closes one when it is not followed by one. The rule is scoped to underscores because Slack applies *bold*, ~strike~ and `code` mid-word. When the closing underscore sits mid-word the span stays literal rather than searching on for a later candidate, which would swallow the text in between. The word-character test runs on whole code points and counts combining marks, so a decomposed "e" plus combining acute and astral letters are not mistaken for non-word characters.
a54e3d9 to
c861a2e
Compare
|
All six confirmed by tracing old vs new output, and all six are now fixed. Pushed as The word-boundary rule is scoped to On the skip-to-a-later-candidate logic: removed rather than repaired. When the closing The word-character test now runs on whole code points via One deliberate difference from On process, you are right that there was no linked issue. Opened #103 with the WHAT/WHY/HOW and linked it from the PR description. It has not been triaged yet, so this is not mergeable until a maintainer adds |
Fixes #103
Summary
parseInlinepaired a marker with the next occurrence of the same character anywhere later in the text, with no check on what surrounded either marker. An underscore inside one identifier or URL (themerge_requestspath segment in a GitLab link) paired with an unrelated underscore much later in the message, italicising everything between them and deleting both underscores, which destroyed both URLs.An underscore now only opens a span when it is not preceded by a word character, and only closes one when it is not followed by one. The rule is scoped to underscores because Slack applies
*bold*,~strike~and`code`mid-word. When the closing underscore sits mid-word the span stays literal rather than searching on for a later candidate, which would swallow the text in between.The word-character test runs on whole code points and counts combining marks, so a decomposed
é(e + U+0301) and astral letters are not mistaken for non-word characters.Behaviour change
One case differs from
mainbeyond the bug itself:_hello_world_is now fully literal, where it previously rendered italichellofollowed by plainworld_. That matches Slack leavingsnake_caseunformatted, and literal is the safe failure mode for a parser whose output cannot be read back before it is sent.*bold*,~strike~and`code`are unchanged in every case, including mid-word (*a*b,a`b`c), doubled markers (**bold**,~~strike~~) and a~inside a URL path followed by a later~strike~span.Test plan
bun test(323 pass, 0 fail) andbun run type-checkclean._italic_still works alongside a laterfile_name, an underscore pair spanning an identifier stays literal, both Unicode forms ofcafé_, an astral letter before_, mid-word bold/strike/code, doubled markers, and a URL~against a later strike span.