fix(markdown): avoid quadratic parse time on long unbroken words

[herdiyana256]

The inline parser adds a plain-text "accelerator" regex that fast-forwards runs of word characters. Both variants require a trailing whitespace:

if (document.hasCustomInlineSyntaxes) {
  syntaxes.add(TextSyntax(r'[A-Za-z0-9]+(?=\s)'));
} else {
  syntaxes.add(TextSyntax(r'[ \tA-Za-z0-9]*[A-Za-z0-9](?=\s)'));
}

When a run of word characters reaches the end of a line without a trailing whitespace, for example a single long word, the greedy */+ consumes the whole run, the (?=\s) lookahead fails, and the engine backtracks one character at a time. The parser then advances one position and repeats the full scan, so parsing is quadratic in the length of the run.

markdownToHtml on a single long word, no spaces:

32,000 chars  : 12,926 ms
16,000 chars  :  3,131 ms
 8,000 chars  :    823 ms

The same character count split into short words is under 1 ms, confirming it is the unbroken run, not the length, that is slow. This is easy to reach with untrusted input (a long URL, a base64 blob, or any unbroken token in user-supplied markdown), so it is a denial-of-service risk for anything rendering markdown it did not author.

The fix lets the run also end at $, so it matches in a single step when it reaches the end of a line:

syntaxes.add(TextSyntax(r'[A-Za-z0-9]+(?=\s|$)'));
...
syntaxes.add(TextSyntax(r'[ \tA-Za-z0-9]*[A-Za-z0-9](?=\s|$)'));

The match content is unchanged. A trailing word with no following whitespace was already emitted as plain text, just one character at a time. After the fix a 2,000,000-character word parses in 35 ms.

The full suite (2763 tests, including the CommonMark and GFM conformance suites) passes unchanged, and a regression test in test/regression_test.dart parses a 200k-character word and asserts it finishes well under the old timings. dart analyze --fatal-infos and dart format --set-exit-if-changed are clean.

[gemini-code-assist]

Code Review

This pull request addresses a performance issue where parsing a long unbroken run of word characters resulted in quadratic execution time. The fix updates the plain-text accelerator regex to allow matching at the end of a line and adds a regression test. The reviewer notes that the regex is still vulnerable to quadratic backtracking if the long word is followed by punctuation (e.g., an exclamation mark) and suggests a more robust, future-proof parsing approach along with an expanded regression test to cover these edge cases.

[gemini-code-assist]

While adding |$ to the lookahead fixes the quadratic backtracking when a long word is at the end of a line or string, the regex still suffers from quadratic backtracking if a long word is followed by punctuation (e.g., a * 200000 + !). Since the lookahead (?=\s|$) fails on !, the engine will backtrack and retry at every single character position, leading to the same quadratic performance issue.

Instead of relying on the assumption that there are no default inline syntaxes starting with a word character (which is not future-proof and could introduce subtle bugs if new syntaxes are added), we should prioritize a robust, future-proof implementation. We can dynamically construct a negated character class based on the active syntaxes' trigger characters, or use a character-by-character scanner to safely consume plain text without backtracking.

References

1. When building foundational components, prioritize robust, future-proof implementations over simpler ones that may introduce subtle bugs or rely on unintended behavior.

[gemini-code-assist]

Let's expand the regression test to cover both the end-of-line case and the followed-by-punctuation case. This ensures that the parser remains immune to quadratic backtracking even when a long word is followed by punctuation.

  test('long unbroken word does not take quadratic time', () {
    // The plain-text accelerator regex required a trailing whitespace, so a
    // long run of word characters that reached the end of a line without one
    // (a single long word) forced the regex to backtrack across the whole run
    // at every position, giving quadratic parse time. A ~200k character word
    // used to take tens of seconds; it should now be effectively instant.
    for (final suffix in [r'\n', '!']) {
      final input = '\n' + 'a' * 200000 + suffix;

      final time = Stopwatch()..start();
      final html = markdownToHtml(input); // Should not hang.
      expect(html, isNotNull); // To use the output.
      expect(time.elapsedMilliseconds, lessThan(10000));
    }
  });

[srawlins]

FYI @lrhn I know you like this kind of thing

[srawlins]

Alright let's land it. If the benchmark test is flaky, we can mark it local (or skip or something).

[srawlins]

CC @natebosch do you know if we have any precedent or guidelines or systems for benchmarks? I'm very very hesitant to check in a test like this which I imagine will likely become flaky on certain CI situations. I'd rather have no benchmark than a flaky benchmark. Can we take advantage of any GitHub CI or LUCI infrastructure?

[herdiyana256]

Happy to make this a plain correctness regression test with no timing assertion, so there's nothing to go flaky on slower CI it would just parse the pathological inputs and assert the output is correct, and rely on a suite-level timeout if the quadratic behavior ever regresses. Or I can mark it local/skip if you'd prefer. Whichever you like and I'll update it.

[srawlins]

We can leave it for now.

[srawlins]

Thanks much!