editor: auto-close brackets and smart indent on Enter

[Muhammad-Owais-Warsi]

Closes #3015

Description

The code editor is missing two standard editing behaviors:

1. Smart indent — pressing Enter after a line ending with {, (, [ or : keeps the same indent instead of adding a level. Entering a newline between a bracket pair ({|}) does not split onto three lines.
2. Auto-close brackets — typing (, [, {, " or ' inserts only the opener. Typing a closer that already follows duplicates it instead of skipping past. Backspace between an empty pair deletes one char instead of both.
   This adds both, enabled by default, with opt-outs following the existing folding/line_number builder + setter pattern:

• .auto_close(bool) / set_auto_close(...)
• .smart_indent(bool) / set_smart_indent(...)
  Quote pairing/skipping is guarded against word characters so contractions like don't are unaffected. Pair deletion only fires on truly empty pairs.

Screenshot

Screen.Recording.2026-09-08.183531.mp4

Break Changes

None. Both features default to enabled; existing callers get the new behavior automatically and can opt out per-editor.

How to Test

cargo run -p example-editor

• Type { → get {} with cursor inside
• Enter between {} → three lines, cursor indented on middle line
• Type } → skips past existing closer, no duplicate
• Backspace between () → both deleted
• Backspace in {d|} → only d deleted, } kept
• Type ' after don → no pairing

Checklist

• I have read the CONTRIBUTING document and followed the guidelines.
• Reviewed the changes in this PR and confirmed AI generated code (If any) is accurate.
• Passed cargo run for story tests related to the changes.
• Tested macOS, Windows and Linux platforms performance (if the change is platform-specific)

[huacnlee]

Please make the editing rules configurable at the Base boundary:

• Editor belongs to gpui-base, which must remain independent of tree-sitter. Expose bracket pairs and their auto-close/newline behavior as Editor parameters instead of hard-coding characters in the editing engine.
• gpui-component should own language configuration and tree-sitter integration, supplying those rules and adapting syntax-context/indentation information to a parser-independent Base interface. Auto-close policy is language metadata; bracket/indent queries provide syntax information. Base should not consume tree-sitter-specific configuration or queries.

Both conveniences fit the editor, but the current universal character rules are not suitable for its language-independent API. The following correctness and performance issues also need addressing (| denotes the cursor):

1. P1 — Unicode panic (crates/base/src/input/editor/mod.rs:243): typing a quote after 中| panics with “byte index is not on a char boundary”. range.start is already the insertion boundary; subtracting one byte can split a Unicode character and also reads the wrong preceding ASCII character. Use a Unicode-safe previous-character lookup. For example, typing an apostrophe after a| also incorrectly produces two apostrophes.
2. P1 — Unrelated text deletion (crates/base/src/input/base/state.rs:1646, crates/base/src/input/editor/mod.rs:224): UTF-8 byte ranges are passed to replace_text_in_range_silent, which accepts UTF-16 ranges. Backspace at 中(|)abc produces 中()c, deleting ab; typing ) there produces 中())bc, deleting a. Convert the ranges or use the byte-oriented editing primitive. Cover both paths with CJK and non-BMP prefixes.
3. P2 — Closing quotes do not skip (crates/base/src/input/editor/mod.rs:215-220): typing a double quote at "hello|" produces "hello"". The word guard for opening quotes must not suppress recognition of a closing quote. Test ordinary strings, contractions and escapes.
4. P2 — Linear work per keypress (crates/base/src/input/base/state.rs:1635,1784 and the quote lookups): slice(..cursor).chars().last() traverses the whole document prefix. An isolated optimized Ropey benchmark measured approximately 2.5 ms at 1 MB and 25 ms at 10 MB for one lookup, before other editor work. Seek to the cursor and traverse backwards.
5. P2 — Smart-indent opt-out is ignored for pairs (crates/base/src/input/base/state.rs:1779-1810): after set_smart_indent(false), Enter at {|} still adds an extra indent level. The branch only checks auto_close. Respect the independent options and test their combinations.

Validation on cb155386c6bdee10df9a16e6a4086f9bc5327a3b: the seven added tests and the input-filtered suite (227 tests) pass, but six additional temporary GPUI context reproductions fail on the cases above. No real-window or local macOS/Windows validation was performed. The separate Docs CI failure concerns a missing generated UI-testing page and is not attributed to these three changed files.

Security: BLOCK for affected text-input availability/integrity, based on the reproduced Unicode panic and deletion of unrelated text; no remote exploit or native memory corruption is claimed.

Please address the findings above, then request my review again.

[Muhammad-Owais-Warsi]

Summary of Changes

Architecture & Policy Refactoring

• Extracted Bracket Policy from Base: Introduced the EditRules { pairs, indent_triggers, auto_close, smart_indent } type.
• Engine Isolation: Engine methods (handle_auto_close, backspace, split, indent_of_next_line_at) now query rules exclusively with zero character literals hardcoded in engine logic. Default rules reside strictly in the Default implementation and can be explicitly overridden.
• Language Rules Synchronization: gpui-component supplies language_rules() with a special-case fallback (text/plain → empty rules), synchronized at render time alongside ensure_highlighter_factory. Explicit edit_rules() calls set a customization flag that sync honors.
• WASM Compatibility: Removed tree-sitter dependencies from Base. The component lookup table uses string matching to handle cross-platform Language enum stub differences seamlessly.
• Single PR Scope: Combines base engine plumbing and component supply into one cohesive change (+548/−157 across 9 files, including ~200 lines of tests) to prevent default policies from remaining de facto rules.

---

Key Fixes & Correctness Improvements

• P1 Unicode Panic Fix: Replaced unsafe char boundary indexing (range.start.saturating_sub(1)) with Rope-native chars_at().reversed() lookups.
  • Covered by: test_auto_close_quote_after_cjk
• P1 UTF-8/UTF-16 Boundary Encoding: Guaranteed all silent-edit ranges translate through range_to_utf16(), constructing byte ranges directly via len_utf8().
  • Covered by: test_backspace_pair_with_cjk_prefix (e.g., 中(|)abc correctly becomes 中|abc without corrupting neighboring characters)
• P2 Quote Skip Logic: Decoupled the open-guard (before cursor) from close-recognition (after cursor). The skip branch is intentionally unguarded since a matching follower acts strictly as a closer (resolves the "hello|" issue).
• P2 Linear Scan Optimizations: Replaced full-prefix string slices (slice(..cursor)) with direct seek-based cursor lookups using chars_at().
• P2 Flag Independence: Line splitting now strictly requires auto_close, while structural extra indentation independently requires smart_indent.
  • Covered by: test_enter_split_respects_smart_indent_off and test_enter_no_split_when_auto_close_off

---

Testing & Verification

• Suite Results: 233 Base input tests + 3 new gpui-component language rule integration tests pass cleanly.
• Formatting: cargo fmt --check passes cleanly.
• Coverage Expansion: Added test cases covering custom delimiters («»), apostrophes/contractions (don't), CJK/non-BMP multi-byte characters, and explicit collapsed-selection assertions (resolving previous selection-extension edge cases via set_cursor_to and set_selected_range).

[huacnlee]

Re-reviewed 168dde418f97a93958819ab4b85f6d214c7b1832. The previous Unicode panic, byte/UTF-16 deletion bugs, ordinary closing-quote skip, prefix scans, and smart-indent opt-out have been addressed. The following blockers remain (| denotes the cursor):

1. P2 — Unconditional notification during rendering (crates/base/src/input/base/state.rs:8223-8225, called from crates/component/src/input/input.rs:384): every Editor render synchronizes rules and calls cx.notify(), even when the rules are identical or customization prevents any update. Three identical sync calls produced three notifications in a context test. This can sustain redundant rendering and observer work while idle; native-window CPU impact was not measured. Make synchronization idempotent and avoid notifying during unchanged render-time synchronization. Cover unchanged and customized rules.

2. P2 — Skipping a closer corrupts undo history (crates/base/src/input/editor/mod.rs:222-227): at (|), type ) and then Undo; the text becomes ()). The initial insertion and subsequent silent deletion are separately recorded: silent replacement suppresses typing hooks, not history. Handle skipping as a cursor move, or a coherent operation that cannot expose the transient duplicate through Undo/Redo. Add history and cursor-position regressions for skip and pair insertion.

3. P2 — Individual option overrides freeze all language rules (crates/base/src/input/base/state.rs:8257-8262, also the auto-close setters): EditorState::new(...).language("text").smart_indent(false) marks the entire rule set customized before Component synchronization. The plain-text rules are consequently ignored, and typing ( produces (). This was reproduced through the actual TextInputState::sync_edit_rules path. Track flag overrides independently of language pairs/triggers; reserve suppression of whole-table synchronization for explicit edit_rules customization. Test builder/setter order and language changes.

4. P2 — Escaped quotes incorrectly skip the string terminator (crates/base/src/input/editor/mod.rs:219-228): starting with "hello\|", typing " should produce "hello\"|"; instead it produces "hello\"|, leaving the only final quote escaped and the string unterminated. The unconditional closer branch does not check escape or syntax context. Apply language-appropriate context before opening/skipping pairs; cover odd/even backslashes, strings and comments.

5. P2 — The prior language/context boundary request is only partially implemented (crates/component/src/input/edit_rules.rs:8-17, crates/base/src/input/editor/edit_rules.rs:16-24): exposing EditRules is progress, but every non-plain language still gets the same Base default table. Component supplies no actual language-specific policy or syntax-context adaptation, and the Base interface cannot express per-pair newline behavior or context decisions. Rust, JSON, HTML and Python therefore still share colon indentation and quote policies. Complete the previously requested architecture: Component owns language metadata and tree-sitter integration, adapting syntax information into parser-independent Base interfaces; Base must remain independent of tree-sitter. Exercise the real language/provider synchronization path, not only the lookup function.

The two conveniences can reduce manual code editing, but require maintained language rules and context handling. Recommendation: retain them only with those boundaries completed. Product decision remains pending separately from this technical review.

Validation: the unmodified Base input suite passed (233 tests); additional temporary Base context reproductions passed 8 and failed 4 on the cases above. The Component input suite passed 29 and failed the additional plain-text/option integration reproduction. Formatting and diff checks passed. No native-window, local macOS/Windows or wasm validation was performed. No new latency benchmark was run.

Security: PASS, scoped to the reviewed Unicode/indexing boundaries and absence of new external I/O; CJK and non-BMP skip/deletion reproductions preserve neighboring text. The undo and escaped-quote defects remain functional correctness blockers. This is not a complete editor security audit.

Please address the findings above, then request my review again.

[Muhammad-Owais-Warsi]

1. Notification Churn Optimization

• ensure_edit_rules now tracks and returns whether rules actually changed.
• notify() fires strictly on genuine state changes, while customized paths return silently to keep idle renders quiet.

2. Undo History Corruption Fix

• Refactored quote skip logic into a pre-insertion cursor move inside replace_text_in_range.
• When a typed closer already follows the cursor (collapsed selection, no IME, non-silent edit), the keystroke is directly consumed via set_cursor_to without mutating text or touching history records.
• Covered by: test_skip_records_no_history (verifies undo bypasses skips to remove prior characters) and test_pair_insert_undo_removes_both (verifies pair insertion coalesces cleanly into a single history entry).

3. Granular Flag Customization & Synchronization

• Whole-table edit_rules() sets rules_customized, bypassing automatic language sync entirely.
• Individual auto_close() and smart_indent() setters set per-flag pins. Language sync merges pairs/triggers from language rules while preserving these user pins (e.g., .language("text").smart_indent(false) successfully loads empty text pairs while holding smart_indent off).
• Covered by: test_flag_pins_survive_language_sync across the real sync path and split state combination tests.

4. Escaped Quote Handling

• Implemented odd-backslash run checks prior to quote evaluation.
• Trailing escaped quotes (e.g., "hello\"| + ") now insert literally, while even backslash runs trigger standard skip behavior.
• Covered by: Odd and even backslash unit test cases.

5. Language Boundary & Tree-Sitter Integration

• Differing language tables configured:
  • JSON: Removed single quotes (') and colon (:) triggers.
  • Rust / HTML: Removed colon (:) triggers.
  • Python: Retains colon (:) triggers.
  • Text / Plain: Empty ruleset (prevents unwanted quote pairing in plain text).
• Architecture: Introduced SyntaxContextProvider trait and holder in Base (zero tree-sitter imports). Added tree-sitter-backed component provider featuring incremental parse reuse, per-language reinstall tracking, and WASM/feature-off fallback paths.
• Render Sync: Render-time sync installs both rules and providers while honoring explicit app overrides.
• Context Awareness: Indentation and bracket-splitting now consult the syntax provider to prevent invalid extra levels or three-way splitting inside strings and comments.
• Covered by: Table unit tests, comment-provider seam tests, 3 real-grammar JSON provider tests (--features tree-sitter), and string-gated indent/split test cases.