stream wordlist

[Mzack9999]

Stream the wordlist and FUZZ expansion directly into the hybrid map instead of buffering the full wordlist times domain product in a slice first, keeping memory bounded on large bruteforce inputs.

Closes #993

Summary by CodeRabbit

• Bug Fixes
  • Improved host expansion so fuzzing and wordlist combinations are processed more efficiently and with fewer duplicates.
  • Simplified single-input handling for more consistent results.
• Tests
  • Added coverage for fuzzed host expansion.
  • Added coverage to ensure duplicate entries in wordlists do not create duplicate expanded hosts.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 431ce3e6-a65b-4817-84ea-8390d6dc4a17

📥 Commits

Reviewing files that changed from the base of the PR and between 1976129 and 3ec0757.

📒 Files selected for processing (2)

• internal/runner/runner.go
• internal/runner/runner_test.go

---

Walkthrough

prepareInput now streams FUZZ and wordlist expansions directly into the hybrid map, deduplicating hosts as they are inserted. The single-host path now reuses the list helper directly, and new tests cover fuzz expansion and streamed wordlist deduplication.

Changes

Runner input expansion flow

Layer / File(s)	Summary
Hybrid-map dedupe helper internal/runner/runner.go	addHostToHMap centralizes hybrid-map lookup and insertion, and the list and channel helpers count only newly added hosts through it.
Streamed FUZZ expansion internal/runner/runner.go, internal/runner/runner_test.go	prepareInput streams FUZZ and wordlist expansions into the hybrid map, uses the direct single-item list path, and adds tests for FUZZ expansion and streamed wordlist deduplication.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

A rabbit hops जहां? No, here I prance 🐇
Through FUZZy fields, I give hosts a chance
I nibble duplicates, one by one
Till hybrid burrows say, “That’s done!”
Tiny nose boop, review complete ✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title matches the main change: streaming wordlist expansion instead of buffering the full product.
Linked Issues check	✅ Passed	The PR removes the full product buffer and streams FUZZ/wordlist expansions into the hybrid map, matching #993.
Out of Scope Changes check	✅ Passed	The code changes and tests stay focused on streaming bruteforce input and deduplication, with no unrelated additions.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch wordlist-stream

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}