fix: correct ansi_c_string regex to handle trailing escaped backslashes

[0x-pankaj]

Problem

The ansi_c_string regex /\$'([^']|\\')*'/ incorrectly parsed ANSI-C quoted strings containing trailing escaped backslashes. For example:

WORDCHARS=$'/-–_:,.;<>~!@#$%^&*|+=[]{}()?\'"`\\'

The old regex [^'] matched any character except ', including \. This meant \\ at the end was consumed one character at a time — the first \ was consumed by [^'], leaving the second \ to also be consumed by [^'], and then ' correctly closed the string. But \\' (escaped backslash followed by closing quote) would fail because \ before ' was consumed by [^'], then \' was treated as an escaped quote instead of letting ' close the string — causing the parser to run past the intended end.

Fix

Changed the regex from:

/\$'([^']|\\')*'/

to:

/\$'([^'\\]|\\.)*'/

By excluding \ from the single-character match [^'\\], every backslash must go through the \\. path, which always consumes backslash + the next character as a pair. This means:

• \\ is consumed as one unit (escaped backslash) — the ' after it stays free to close the string
• \' is consumed as one unit (escaped quote) — correctly handled
• \n, \t, etc. all work correctly too

Tests

• All 100 existing corpus tests pass
• Added new test cases to test/corpus/literals.txt for:
  • WORDCHARS assignment with trailing \\ (the original bug trigger)
  • Escaped backslash at end of string
  • Tab escape \t
  • Mixed escapes with \' and \\ together

Fixes zed-industries/zed#51021