Skip to content
kochetkov-maPublic

About

Fast, deterministic document-to-Markdown conversion for agent systems. Easy to install. No model weights, OCR or external services.

Resources

Stars

9 stars

Watchers

0 watching

Forks

Repository files navigation

brewdoc

PyPI CI Release MIT

Fast, deterministic document-to-Markdown conversion for agent systems. Easy to install. No model weights, OCR or external services.

Local conversion runs offline after installation, with no runtime component downloads. Supported documents become Markdown and a JSON receipt with independent source inspection. See formats and limits. Version 0.4.1 added selected ZIP/XML reader budgets for DOCX/PPTX and workbook metadata/artifacts. Excessive work refuses before output writes; native workbook values and total process memory remain outside those bounds.

Version 0.5.0 added ODT/ODP text and tables, selected MIME email bodies and exact UTF-8 Markdown/text pass-through. It preserved workbook Unicode and exposed datetime precision, and selected one DOCX compatibility fallback. It also added textless PDF page facts and selected caption, table and math-spacing fixes. Version 0.6.0 writes cleaner Markdown with minimal escaping, adds --body-only, accepts documents through --url, decodes legacy HTML charsets and falls back to the whole page in article mode. Version 0.7.0 added --render-js auto and HTML --no-links, refuses captcha, login and JavaScript shell pages with a plain reason, reports HTTP status, redirects and charset in URL receipts, decodes mislabelled HTML charsets by fallback and writes code and image links as standard Markdown links. Version 0.7.1 checks --url TLS against the operating system trust store, converts the title, description, JSON-LD and noscript text of some JavaScript shell pages as a partial result, and names AWS WAF challenges. Version 0.8.0 adds static inspection for all 15 supported extensions, exact payload retrieval, and source findings alongside readable text. It preserves scripts as literal data without running them.

Install

These examples select 0.8.0. Use a channel listed as available in that release. Choose one method:

python -m pip install "brewdoc==0.8.0"
uv tool install --python 3.12 "brewdoc==0.8.0"
pipx install --python 3.12 "brewdoc==0.8.0"

Use a virtual environment for pip. Python 3.12+ is required; local and URL HTML need CPython 3.12-3.14. INSTALL.md covers prerequisites, Windows, uvx, PATH, update and removal. Latest release lists actual assets and channel availability.

Standalone downloads

Use downloads listed as available in the release. Python and dependencies are bundled. Verify SHA256SUMS and keep the entire extracted directory.

Linux x64 download Windows x64 download macOS ARM64 download

Native target Verification environment JavaScript
Linux x64/glibc Ubuntu 22.04, glibc 2.35 Bundled
Windows x64 Windows Server 2025 Refused before requests
macOS ARM64 macOS 15, Apple Silicon Bundled

Older OSes, Linux musl/ARM64 and macOS Intel are outside this binary set. Windows is unsigned; macOS has an ad hoc signature without Developer ID/notarization. Linux archives have checksums, without a detached publisher signature. Download and launch.

npm

npm

npm targets Linux x64/glibc and macOS ARM64. Windows uses the GitHub native download or Python package. Use npm/npx only when listed as available in the release. INSTALL.md contains exact-version installation, update and removal instructions.

Homebrew

Use kochetkov-ma/brew/brewdoc only when the release lists it as available. It installs the complete native bundle on macOS 15+ ARM64 or Linux x64 with system glibc 2.35+. See installation and lifecycle.

Usage

brewdoc file.pdf --out file.md
brewdoc --body-only paper.pdf --out paper.md
brewdoc book.epub --out book.md
brewdoc file.xlsx --sheet Calc --sheet Inputs --out book.md
brewdoc file.xlsm --artifact formula/sheet/000002=formulas.json
brewdoc file.xlsm --artifact vba/project/000001=vbaProject.bin
brewdoc file.pdf --artifact inspection/report=inspection.json --out file.md
brewdoc --self-check

--sheet is repeatable and workbook-only. Sheet names are exact and case-sensitive; caller order retains full-workbook ordinals. --artifact is repeatable on every supported route. Use inspection/report for the full report; exact payload keys come from its artifact inventory. Requests and output collisions are validated before writes. Paths resolve against the caller's directory; --out creates parents. Readable refusal preserves existing Markdown; requested inspection artifacts can still be written. Self-check is offline and has its own output. --body-only drops the readable title, metadata, omissions, contents and unit anchors; unit headings stay only for several units. Source findings and the receipt remain. See body-only output.

Markdown and receipts

Parsed document output uses brewdoc.markdown/2: quoted source title, Metadata, Artifacts, Known omissions, Contents, then anchored units. Independent findings follow under Links, Active content and Extracted content when present. PDF uses page/000001, workbooks sheet/000001, PPTX/ODP slide/000001, and DOCX/EPUB/HTML/ODT/EML chapter/000001. Empty declared units remain navigable. Escaping is minimal; HTML, EPUB and EML omit form controls from their readable projection and write images with alt text as ![alt](src). Omission notices describe that projection; inspection can recover additional original content. See static inspection and Markdown escaping. Metadata holds source/selection/content hashes, conversion tallies and artifact capabilities; it excludes paths, timestamps, permissions and host data. EPUB retains declared publication metadata.

UTF-8 .md and .txt use brewdoc.markdown/passthrough/1: exact source bytes with no generated envelope, contents or anchors. BOM, whitespace, newline variants and absent final newline survive; empty input succeeds. The receipt reports route text, kind none and zero units. Without --out, the JSON receipt precedes the unchanged body bytes.

Conversion prints one JSON receipt line first. With --out, Markdown goes to the file; without it, Markdown follows on stdout. Local receipts use brewdoc.receipt/1; URL receipts use brewdoc.receipt/2.

Receipt fields Meaning
file_ok, reason Conversion success or named refusal
receipt_schema Local brewdoc.receipt/1 or URL brewdoc.receipt/2
route, source, unit_kind, units Reader, local filename/redacted URL, content kind/count
tables, text_regions, columns_split Conversion tallies
dropped, broken_ligature_words Counted normalization; malformed font words are folded, not guessed
not_carried Readable projection omissions; inspection has separate coverage
inspection Source identity, capability status counts and report artifact key
out Markdown path, or null for stdout
markdown_schema, unit_keys Success-only schema and ordered readable source keys
artifacts Available/requested artifacts, including inspection on readable refusal
selected_sheets Caller-ordered names, only when selected
article_fallback, charset_fallback Local HTML whole-page fallback and charset chosen by fallback, only when used
acquisition URL provenance, HTTP status, redirects, challenge, charset, frozen snapshot and capture status

Exit 0 means conversion, 1 refusal and 2 invalid usage. Success does not prove content fidelity or complete JavaScript. run and the CLI emit UTF-8 without BOM, with LF newlines for generated output. The direct render_pdf, render_doc and render_presentation wrappers retain their ASCII projection. Text pass-through retains the source UTF-8 encoding and newlines. CLI receipt JSON keeps ASCII escaping. The same local file or frozen snapshot produces identical bytes; live pages can change.

Python API

import brewdoc

code, receipt, markdown = brewdoc.run("book.epub", out="book.md")
markdown, tally = brewdoc.render_html("page.html")
refs = brewdoc.list_book_artifacts("book.xlsm", sheets=("Calc",))
payload = brewdoc.read_book_artifact("book.xlsm", refs[0].key, sheets=("Calc",))
report = brewdoc.inspect_document("file.pdf")
refs = brewdoc.list_document_artifacts("file.pdf")
report_bytes = brewdoc.read_document_artifact("file.pdf", "inspection/report")

run accepts sheets, an artifact_outputs key-to-path mapping, article, body_only and links. Renderers include render_pdf, render_doc, render_presentation, render_epub, render_html and render_book. read_formulas provides typed formulas; self_check verifies installed routes. Local APIs never acquire URLs. The document inspection APIs read the original source independently of these renderers. To retrieve a payload, pass its exact listed key to read_document_artifact. Existing workbook keys and APIs remain available.

Explicit URL acquisition

--url deliberately acquires a public HTTP(S) page or supported document before local conversion. Content-Type and magic bytes pick the route: HTML, PDF, Office, OpenDocument, EPUB, EML or text.

brewdoc --url https://example.com/article --out article.md
brewdoc --url https://arxiv.org/pdf/1706.03762 --out attention.md
brewdoc --url https://example.com/article --render-js --timeout 30 --out article.md
brewdoc --url "https://hn.algolia.com/?q=rust" --render-js auto --out search.md
brewdoc --url https://example.com/article --no-links --out article.md

Bare --render-js means always; --render-js auto runs JS only when the static page is a JavaScript shell. Prefer --render-js auto for unknown pages: always runs JS on every page and can hit an engine quota on a page that static mode reads. Pages that need more than the bounded engine refuse, such as crates.io. A shell page whose static HTML embeds at least 200 characters of JSON-LD or noscript text converts that text as a partial result, such as an Airbnb listing or a SoundCloud track; the receipt keeps a javascript_required error. Captcha, bot challenge, login and other shell pages refuse at stage challenge or javascript_required with a plain reason; the receipt names the challenge and keeps the HTTP status. --no-links (HTML only) keeps link text and omits destinations from the readable projection. Default inspection findings and inspection/report retain the original destinations. --timeout is one positive finite waiting budget, default 30 seconds; cleanup/conversion can add time. Public destinations, DNS, redirects, TLS, requests and bytes are validated. Ordinary requests use GET; JS admits only the narrow credential-free HN Algolia XHR POST exception. Local HTML never fetches or runs scripts. Optional JS needs installed/bundled prerequisites, refuses unsupported runtimes before HTTP and downloads no engine. Timing-only expiry can preserve valid partial content; hard failures refuse output. Full URL/JS limits.

Article mode

Introduced in 0.4.0; use a channel listed as available in the release. For Python JS capture, install brewdoc[render-js]==0.8.0; supported standalone bundles include its prerequisites.

brewdoc page.html --article --out article.md
brewdoc --url https://example.com/article --article --out article.md
brewdoc --url https://example.com/article --render-js --article --out article.md
code, receipt, markdown = brewdoc.run("page.html", article=True)
markdown, tally = brewdoc.render_html("page.html", article=True)

Article mode keeps visible main (or role main) content, or outer article content when no main exists; articles outside a main are dropped when one exists. Page-chrome landmarks are skipped, but a header inside an article or a nav inside main stays. Peer articles and multiple mains render in DOM order. When no candidate renders, the whole clean page renders and the receipt adds article_fallback: true; selection never refuses for page structure. There is no text scoring. Selection uses the recovered static HTML or frozen JS snapshot, without layout or inferred author/date metadata. The source title is unchanged.

When content is selected, the receipt's not_carried and Markdown's Known omissions declare: body content outside the selected article or main subtree; complete page coverage is not claimed. The article contract defines selection, fallback and limits.

Earlier source-package CI passed on Ubuntu 22.04 x64/glibc with CPython 3.13.15, and macOS 15 ARM64/Windows Server 2025 x64 with CPython 3.14.7. Each passed four independent wheel/sdist base/optional installs; optional JS executed on Unix and refused before requests on Windows. These are selected tested patches, not a claim about newer versions. These results predate the 0.5.0 changes. CPython 3.12.14 retains the local macOS full-suite baseline and existing CI smoke checks. Package evidence is separate from standalone binaries; other OS/minor combinations, free-threaded CPython and other architectures remain untested here. See package compatibility.

Format contracts | Corpus | Measurements | Results Releasing | Agent entry point | MIT license

Related projects

  • BrewPage: free hosting for HTML, Markdown, JSON and files; each upload returns a shareable HTTPS link.
  • BrewPage MCP server: npm package brewpage-mcp for publishing to BrewPage from MCP clients.
  • claude-brewcode: plugin suite for Claude Code, with documentation. Its brewdoc plugin is a different tool from this brewdoc CLI.
  • brewcode.app: home site of the Brew projects.

About

Fast, deterministic document-to-Markdown conversion for agent systems. Easy to install. No model weights, OCR or external services.

Resources

Stars

9 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages