Preserve the majority input line-ending style in formatter output

[coderabbitai]

Summary

Emit formatter output with the majority line-ending style from the input file.

mdtablefix currently emits Unix LF line endings regardless of the input file line endings.

Rationale

A formatter must not create a line-ending-only diff when the input file uses CRLF line endings. This behavior causes unnecessary changes in Windows-oriented repositories and prevents check-only formatting gates from comparing formatter output directly with valid CRLF source files.

Required changes

• Detect the line endings in each input Markdown file.
• Select the line-ending style that has the majority of line-ending occurrences in that file.
• Emit formatted output with the selected line-ending style.
• Preserve existing Markdown-formatting behavior.
• Define and document deterministic behavior when no line-ending style has a majority.

Affected areas

• Markdown input reading and output serialization.
• In-place formatting behavior.
• Formatter documentation, if it describes line-ending behavior.
• Automated tests for LF, CRLF, mixed-line-ending, and no-majority input.

Acceptance criteria

• A file with a majority of LF line endings produces LF output.
• A file with a majority of CRLF line endings produces CRLF output.
• A mixed-line-ending file produces output that uses its majority line-ending style.
• Tests cover both standard output and in-place formatting, where supported.
• Tests define the result for input with no majority line-ending style.
• Existing Markdown formatting behavior remains unchanged apart from output line-ending selection.

Backlinks

• Originating pull request: Enforce Markdown formatting in check-fmt netsuke#619
• Originating comment: Enforce Markdown formatting in check-fmt netsuke#619 (comment)
• Requested by: @leynos

[leynos]

@CodeRabbit plan

[coderabbitai]

Coding Plan

Summary

Add a backend-independent LineEnding type, a detect_line_ending function, and a serialize_lines helper in src/io.rs, and re-export them from src/lib.rs.

Detect the majority terminator from the raw input at all three I/O boundaries, keep the internal Vec<String> pipeline LF-only, and serialise output with the detected terminator.

Use LF as the deterministic default for ties and for input with no line endings.

Document the decision in a new ADR, the guides, the layout document, and the changelog, and add byte-exact tests for LF, CRLF, mixed, and no-majority input across stdin, file-to-stdout, and in-place modes.

Design Choices

Design Choice 1: Which line-ending style wins when no style holds a majority?

Options Considered:

1. Prefer LF on any tie or absence of line endings.
2. Prefer CRLF on any tie.
3. Prefer the first line-ending style seen in the file.

Chosen Option: 1

Rationale: LF is the current default, it is the safest cross-platform choice, and it produces the smallest diff for the common no-ending or LF-leaning case.

Design Choice 2: Where does the detection helper live, and what is its visibility?

Options Considered:

1. A new LineEnding enum plus free functions in src/io.rs, re-exported from src/lib.rs.
2. A new dedicated module src/line_ending.rs, re-exported from src/lib.rs.

Chosen Option: 1

Rationale: io.rs is already documented as the I/O boundary and file-helpers module; placing detection and serialisation there keeps the read/split/join concerns together and avoids a new module for a small mechanism.

💡 User Tips

Regenerate the plan with different choices with @coderabbitai <feedback>.

Implementation Steps

Phase 1: Add shared line-ending detection and serialisation

This phase introduces the core, backend-independent logic. It detects the majority line-ending style from raw input text and serialises processed lines back with that style. It then applies this logic to the library file-rewrite boundary and covers it with unit tests. The internal Vec<String> pipeline stays LF-normalised and terminator-free; no parsing code changes.

Task 1: Define the LineEnding type and detection function

Add a small type and detector to src/io.rs.

• Add a public LineEnding enum in src/io.rs with two variants for LF and CRLF, and a method that returns the terminator string ("\n" or "\r\n").
• Add a public detect_line_ending(text: &str) -> LineEnding function.
• Count CRLF occurrences (\r\n) and LF-only occurrences (total \n minus CRLF) in the raw input.
• Return CRLF when CRLF occurrences strictly exceed LF-only occurrences; otherwise return LF.
• Treat a tie, and text with no line endings, as LF, per the chosen tie-break rule.
• Re-export LineEnding and detect_line_ending from src/lib.rs so the binary crate in main.rs can use them.
• Write Rustdoc that states the majority rule, the LF tie-break, and the no-ending default.

Task 2: Add a serialisation helper for processed lines

Centralise the "join lines with the chosen terminator" logic.

• Add a public helper in src/io.rs, for example serialize_lines(lines: &[String], ending: LineEnding) -> String.
• Preserve the existing empty-input contract: return an empty String when the processed line vector is empty, and otherwise join with the chosen terminator plus one trailing terminator.
• Re-export this helper from src/lib.rs for reuse by main.rs.

Task 3: Preserve line endings in rewrite_with

Update the library file-rewrite boundary in src/io.rs.

• In rewrite_with, call detect_line_ending on the raw string read by fs::read_to_string before splitting with str::lines().
• Split lines with str::lines() as today, so internal lines remain terminator-free.
• Build the output with serialize_lines using the detected line ending instead of the hardcoded "\n" join.
• Confirm rewrite and rewrite_no_wrap inherit the new behaviour with no signature change.
• Update the Rustdoc on rewrite, rewrite_no_wrap, and rewrite_with to state that they preserve the majority input line-ending style.

Task 4: Add unit tests for detection and rewrite preservation

Cover the new logic in the #[cfg(test)] mod tests block of src/io.rs.

• Add direct tests for detect_line_ending covering pure LF, pure CRLF, CRLF-majority mixed, LF-majority mixed, exact tie (expect LF), and no line endings (expect LF).
• Add rewrite round-trip tests that write a CRLF fixture and a mixed-majority-CRLF fixture, run rewrite, and assert the output bytes retain the expected terminator.
• Add a test that confirms LF input still produces LF output.
• Preserve the existing rewrite_empty_file_no_extra_newline behaviour.

🤖 Prompt for AI agents

Implement the shared, backend-independent line-ending logic in `src/io.rs` and
cover it with unit tests. Goal: detect the majority line-ending style of raw
input and serialise processed lines back using that same style, without altering
any Markdown transformation logic.

- Define a public `LineEnding` enum in `src/io.rs` with two variants (LF and
CRLF), plus a method returning the terminator string (`"\n"` or `"\r\n"`).
- Implement `detect_line_ending(text: &str) -> LineEnding`: count CRLF
occurrences (`\r\n`) and LF-only occurrences (total `\n` minus CRLF count).
Return CRLF only when CRLF occurrences strictly exceed LF-only occurrences;
otherwise return LF. Treat exact ties and text with no line endings as LF.
- Add Rustdoc on `detect_line_ending` stating the majority rule, the LF
tie-break, and the no-ending default.
- Implement `serialize_lines(lines: &[String], ending: LineEnding) -> String`:
return an empty `String` for an empty input vector; otherwise join lines with
the chosen terminator and append one trailing terminator, matching the existing
join contract.
- Re-export `LineEnding`, `detect_line_ending`, and `serialize_lines` from
`src/lib.rs` so `main.rs` can use them.
- Update `rewrite_with` in `src/io.rs`: call `detect_line_ending` on the raw
string from `fs::read_to_string` before calling `str::lines()`. Keep
`str::lines()` splitting as-is so internal lines stay terminator-free. Replace
the hardcoded `"\n"` join with `serialize_lines` using the detected ending.
Confirm `rewrite` and `rewrite_no_wrap` inherit this with no signature change.
Update Rustdoc on `rewrite`, `rewrite_no_wrap`, and `rewrite_with` to state that
the majority input line-ending style is preserved.
- Add unit tests in the existing `#[cfg(test)] mod tests` block in `src/io.rs`:
- Direct tests for `detect_line_ending`: pure LF, pure CRLF, CRLF-majority
mixed, LF-majority mixed, exact tie (expect LF), and no line endings (expect
LF).
- Rewrite round-trip tests: write a CRLF fixture and a mixed-majority-CRLF
fixture, run `rewrite`, and assert output bytes retain the expected terminator.
  - A test confirming LF input still produces LF output.
  - Preserve the existing `rewrite_empty_file_no_extra_newline` test behaviour.

Phase 2: Apply preservation across the CLI I/O paths

This phase brings the CLI binary to parity with the library. It applies the same detection and serialisation to the file mode and the stdin/stdout mode, so --in-place, file-to-stdout, and stdin-to-stdout all preserve the majority line-ending style.

Task 1: Preserve line endings in format_to_string

Update the CLI file boundary in src/main.rs.

• In format_to_string, call detect_line_ending on the raw string returned by directory.read_to_string before splitting.
• Split lines with str::lines() as today.
• Replace the inline fixed.join("\n") + "\n" logic with serialize_lines using the detected line ending.
• Confirm rewrite_in_place inherits the behaviour, because it delegates output construction to format_to_string.
• Confirm the property test formatting_matches_in_place_output still holds, because both stdout and in-place paths share format_to_string.

Task 2: Preserve line endings in the stdin/stdout branch

Update the stdin branch inside main() in src/main.rs.

• Call detect_line_ending on the full stdin string before splitting with str::lines().
• Replace println!("{}", fixed.join("\n")) with a write of serialize_lines(&fixed, ending) to stdout, so the emitted terminator matches the detected style.
• Confirm the trailing-terminator behaviour matches the file paths, so stdin/stdout output stays consistent with file output.

🤖 Prompt for AI agents

Bring the CLI binary in `src/main.rs` to parity with the library by applying the
same detection and serialisation logic to the file mode and the stdin/stdout
mode. Goal: ensure `--in-place`, file-to-stdout, and stdin-to-stdout all
preserve the majority line-ending style detected in Phase 1.

- In `format_to_string`: call `detect_line_ending` on the raw string returned by
`directory.read_to_string` before splitting. Split lines with `str::lines()` as
today. Replace the inline `fixed.join("\n") + "\n"` logic with `serialize_lines`
using the detected line ending. Confirm `rewrite_in_place` inherits this
behaviour automatically, because it delegates output construction to
`format_to_string`. Confirm the property test
`formatting_matches_in_place_output` still holds, because both stdout and
in-place paths share `format_to_string`.
- In the stdin branch of `main()`: call `detect_line_ending` on the full stdin
string before splitting with `str::lines()`. Replace `println!("{}",
fixed.join("\n"))` with a write of `serialize_lines(&fixed, ending)` to stdout,
so the emitted terminator matches the detected style. Confirm the
trailing-terminator behaviour matches the file paths, so stdin/stdout output
stays consistent with file output.

Phase 3: Documentation and integration tests

This phase records the decision, updates user- and developer-facing documentation, and adds byte-exact integration tests across all I/O modes.

Task 1: Add an architecture decision record

Record the majority-vote rule and the LF tie-break.

• Add docs/adrs/0006-line-ending-detection.md following the existing ADR structure: Status, Date, Context, Decision, Consequences, and Options considered.
• State the majority-vote algorithm, the LF tie-break, and the no-ending default in the Decision section.
• Record the deliberate choice to preserve endings only at the I/O boundaries and to keep the internal pipeline LF-only in Consequences.

Task 2: Update the guides and layout documents

Document the user-visible and internal behaviour.

• Add a subsection to docs/users-guide.md that states the tool preserves the majority input line-ending style and describes the no-majority default.
• Add a note to docs/developers-guide.md near the format_to_string trailing-newline note, describing where detection and serialisation occur.
• Extend the io prose or diagram note in docs/architecture.md to include line-ending detection and preservation as an io responsibility.
• Promote src/io.rs to its own bullet in docs/repository-layout.md, describing its read/write and line-ending responsibilities.

Task 3: Add a CHANGELOG entry

Record the change under the pending release.

• Add a single bullet under ## [Unreleased] in CHANGELOG.md, in the ### Added or ### Fixed sub-section.
• Describe the preserved majority line-ending behaviour in imperative present tense.
• Reference the originating issue link if a tracking issue number is available.

Task 4: Add byte-exact integration tests across I/O modes

Add tests that assert exact terminator bytes, because String-based helpers strip \r.

• Follow the byte-exact pattern in tests/cli_matrix/support.rs: stage a fixture with fs::write(path, b"...\r\n..."), run the CLI, then read back with fs::read(path) and assert Vec<u8> equality.
• Add stdin/stdout tests that pipe CRLF and mixed-majority input and assert the exact terminator bytes on output.stdout.
• Add --in-place tests for LF, CRLF, CRLF-majority mixed, and no-majority input, asserting the file bytes retain the expected terminator.
• Add a file-to-stdout test to confirm parity with the in-place mode.
• Add an explicit no-majority test that asserts LF output, matching the documented tie-break.
• Do not route CRLF fixtures through include_lines! or .lines(), because those strip \r.

🤖 Prompt for AI agents

Record the design decision in documentation and add byte-exact integration tests
covering all I/O modes. Goal: make the majority line-ending rule discoverable to
users and developers, and prove it holds at the byte level.

- Add `docs/adrs/0006-line-ending-detection.md` following the existing ADR
structure (Status, Date, Context, Decision, Consequences, Options considered).
State the majority-vote algorithm, the LF tie-break, and the no-ending default
in the Decision section. Record in Consequences that endings are preserved only
at the I/O boundaries and that the internal pipeline stays LF-only.
- Update `docs/users-guide.md` with a subsection stating that the tool preserves
the majority input line-ending style, and describing the no-majority default.
- Update `docs/developers-guide.md` with a note near the `format_to_string`
trailing-newline note, describing where detection and serialisation occur.
- Extend the `io` prose or diagram note in `docs/architecture.md` to include
line-ending detection and preservation as an `io` responsibility.
- Promote `src/io.rs` to its own bullet in `docs/repository-layout.md`,
describing its read/write and line-ending responsibilities.
- Add a single bullet under `## [Unreleased]` in `CHANGELOG.md` (in `### Added`
or `### Fixed`), describing the preserved majority line-ending behaviour in
imperative present tense, and reference the originating issue link if available.
- Add byte-exact integration tests following the pattern in
`tests/cli_matrix/support.rs`: stage fixtures with `fs::write(path,
b"...\r\n...")`, run the CLI, read back with `fs::read(path)`, and assert
`Vec<u8>` equality. Do not use `include_lines!` or `.lines()` for CRLF fixtures,
because they strip `\r`.
- Add stdin/stdout tests piping CRLF and mixed-majority input, asserting exact
terminator bytes on `output.stdout`.
- Add `--in-place` tests for LF, CRLF, CRLF-majority mixed, and no-majority
input, asserting file bytes retain the expected terminator.
  - Add a file-to-stdout test to confirm parity with the in-place mode.
- Add an explicit no-majority test asserting LF output, matching the documented
tie-break.

Research

mdtablefix is a line-oriented Markdown formatter. Input text is split into a Vec<String> at the outer I/O boundary, processed by pure Vec<String> → Vec<String> transforms in process.rs, wrap.rs, and textproc.rs, then rejoined into a single output string. There are three independent read/rejoin boundaries, and all three normalise line endings to LF today:

• src/io.rs::rewrite_with — library file rewrite via std::fs.
• src/main.rs::format_to_string — CLI file mode via cap_std::fs_utf8::Dir (also used by rewrite_in_place for --in-place).
• src/main.rs::main() stdin branch — reads stdin, prints with println!.

Every read uses str::lines(), which discards \r. Every write uses fixed.join("\n") + "\n". No module tracks the original terminator. The internal pipeline assumes terminator-free LF lines, so \r must not leak into Vec<String>. Line-ending detection and emission must live only at the three I/O boundaries.

---

🚀 Next Steps

🤖 All AI agent prompts combined

Task: 1

Implement the shared, backend-independent line-ending logic in `src/io.rs` and
cover it with unit tests. Goal: detect the majority line-ending style of raw
input and serialise processed lines back using that same style, without altering
any Markdown transformation logic.

- Define a public `LineEnding` enum in `src/io.rs` with two variants (LF and
CRLF), plus a method returning the terminator string (`"\n"` or `"\r\n"`).
- Implement `detect_line_ending(text: &str) -> LineEnding`: count CRLF
occurrences (`\r\n`) and LF-only occurrences (total `\n` minus CRLF count).
Return CRLF only when CRLF occurrences strictly exceed LF-only occurrences;
otherwise return LF. Treat exact ties and text with no line endings as LF.
- Add Rustdoc on `detect_line_ending` stating the majority rule, the LF
tie-break, and the no-ending default.
- Implement `serialize_lines(lines: &[String], ending: LineEnding) -> String`:
return an empty `String` for an empty input vector; otherwise join lines with
the chosen terminator and append one trailing terminator, matching the existing
join contract.
- Re-export `LineEnding`, `detect_line_ending`, and `serialize_lines` from
`src/lib.rs` so `main.rs` can use them.
- Update `rewrite_with` in `src/io.rs`: call `detect_line_ending` on the raw
string from `fs::read_to_string` before calling `str::lines()`. Keep
`str::lines()` splitting as-is so internal lines stay terminator-free. Replace
the hardcoded `"\n"` join with `serialize_lines` using the detected ending.
Confirm `rewrite` and `rewrite_no_wrap` inherit this with no signature change.
Update Rustdoc on `rewrite`, `rewrite_no_wrap`, and `rewrite_with` to state that
the majority input line-ending style is preserved.
- Add unit tests in the existing `#[cfg(test)] mod tests` block in `src/io.rs`:
- Direct tests for `detect_line_ending`: pure LF, pure CRLF, CRLF-majority
mixed, LF-majority mixed, exact tie (expect LF), and no line endings (expect
LF).
- Rewrite round-trip tests: write a CRLF fixture and a mixed-majority-CRLF
fixture, run `rewrite`, and assert output bytes retain the expected terminator.
  - A test confirming LF input still produces LF output.
  - Preserve the existing `rewrite_empty_file_no_extra_newline` test behaviour.
===============================================================================

Task: 2

Bring the CLI binary in `src/main.rs` to parity with the library by applying the
same detection and serialisation logic to the file mode and the stdin/stdout
mode. Goal: ensure `--in-place`, file-to-stdout, and stdin-to-stdout all
preserve the majority line-ending style detected in Phase 1.

- In `format_to_string`: call `detect_line_ending` on the raw string returned by
`directory.read_to_string` before splitting. Split lines with `str::lines()` as
today. Replace the inline `fixed.join("\n") + "\n"` logic with `serialize_lines`
using the detected line ending. Confirm `rewrite_in_place` inherits this
behaviour automatically, because it delegates output construction to
`format_to_string`. Confirm the property test
`formatting_matches_in_place_output` still holds, because both stdout and
in-place paths share `format_to_string`.
- In the stdin branch of `main()`: call `detect_line_ending` on the full stdin
string before splitting with `str::lines()`. Replace `println!("{}",
fixed.join("\n"))` with a write of `serialize_lines(&fixed, ending)` to stdout,
so the emitted terminator matches the detected style. Confirm the
trailing-terminator behaviour matches the file paths, so stdin/stdout output
stays consistent with file output.
===============================================================================

Task: 3

Record the design decision in documentation and add byte-exact integration tests
covering all I/O modes. Goal: make the majority line-ending rule discoverable to
users and developers, and prove it holds at the byte level.

- Add `docs/adrs/0006-line-ending-detection.md` following the existing ADR
structure (Status, Date, Context, Decision, Consequences, Options considered).
State the majority-vote algorithm, the LF tie-break, and the no-ending default
in the Decision section. Record in Consequences that endings are preserved only
at the I/O boundaries and that the internal pipeline stays LF-only.
- Update `docs/users-guide.md` with a subsection stating that the tool preserves
the majority input line-ending style, and describing the no-majority default.
- Update `docs/developers-guide.md` with a note near the `format_to_string`
trailing-newline note, describing where detection and serialisation occur.
- Extend the `io` prose or diagram note in `docs/architecture.md` to include
line-ending detection and preservation as an `io` responsibility.
- Promote `src/io.rs` to its own bullet in `docs/repository-layout.md`,
describing its read/write and line-ending responsibilities.
- Add a single bullet under `## [Unreleased]` in `CHANGELOG.md` (in `### Added`
or `### Fixed`), describing the preserved majority line-ending behaviour in
imperative present tense, and reference the originating issue link if available.
- Add byte-exact integration tests following the pattern in
`tests/cli_matrix/support.rs`: stage fixtures with `fs::write(path,
b"...\r\n...")`, run the CLI, read back with `fs::read(path)`, and assert
`Vec<u8>` equality. Do not use `include_lines!` or `.lines()` for CRLF fixtures,
because they strip `\r`.
- Add stdin/stdout tests piping CRLF and mixed-majority input, asserting exact
terminator bytes on `output.stdout`.
- Add `--in-place` tests for LF, CRLF, CRLF-majority mixed, and no-majority
input, asserting file bytes retain the expected terminator.
  - Add a file-to-stdout test to confirm parity with the in-place mode.
- Add an explicit no-majority test asserting LF output, matching the documented
tie-break.

💡 Iterate on the plan with: `@coderabbitai` <feedback>

Example Feedback
- `@coderabbitai` You can skip phase 3. Add a simple unit test case for phase 2.
- `@coderabbitai` For design choice 1 go ahead with option 3 and replan.

---