Swap tokenization to shlex; add globset and shellexpand

[exlier]

Summary: Replace the hand-rolled lexer with shlex-based tokenization to simplify quoting/escape handling, and add globset + shellexpand for tilde/env and glob expansion.

Files changed: Cargo.toml (added shlex, globset, shellexpand), src/parser.rs (new shlex-based tokenize_line, split_punctuation and expand_and_glob_arg; parser AST left unchanged).

Notes:

• split_punctuation handles attached punctuation (>, >>, |, <, <<, ==, !=, &>, 2>&1, etc.) so existing parser behavior remains compatible without requiring whitespace in all cases.
• Glob expansion is conservative/non-recursive and expands matches in the current working directory; unmatched patterns are returned as-is.

Summary by CodeRabbit

• Chores
  • Added internal support for improved command parsing, pattern matching, and shell-style path expansion.
  • No user-facing behavior or interface changes in this release.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 5ff93a6b-a452-41b5-a05e-2d5bfbfd1e2d

📥 Commits

Reviewing files that changed from the base of the PR and between ae7e622 and 9599d25.

📒 Files selected for processing (1)

• Cargo.toml

---

📝 Walkthrough

Walkthrough

The pull request adds shlex, globset, and shellexpand to Cargo.toml for shell tokenization, glob matching, and shell-style expansion.

Changes

Shell processing dependencies

Layer / File(s)	Summary
Declare shell processing dependencies Cargo.toml	Adds shlex, globset, and shellexpand dependencies.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Suggested reviewers: elianthorne

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}