feat!: macOS support, scanner hardening, and bundled Qt app (v2.0)#23
Merged
JeanExtreme002 merged 34 commits intoMay 25, 2026
Merged
Conversation
Major release adding native macOS support, fixing latent bugs in the Windows/Linux backends, and tightening cross-platform robustness. See CHANGELOG.md for the full list. Added - macOS backend via Mach VM APIs (task_for_pid, mach_vm_read_overwrite, mach_vm_write, mach_vm_region, mach_vm_protect). Self-process works without entitlements; cross-process needs com.apple.security.cs.debugger or SIP off + root. - snapshot_memory_regions() + memory_regions= kwarg on search_by_value* and search_by_addresses for refine-scan workflows. - bufflength is now optional for numeric types (int->4, float->8, bool->1). - iter_region_chunks reads multi-GB regions in 256MB chunks across all three backends (prevents OOM in browser/JVM-sized targets). - PyMemoryEditorError base + AmbiguousProcessNameError. - py.typed marker; mypy + pytest-cov in CI. Fixed (critical) - Platform detection: "win" in sys.platform matched darwin, breaking macOS imports. - Read/Write/process_vm_*v now check argtypes/return value; failed reads no longer return zeroed buffers indistinguishable from real data. - scan_memory off-by-one (skipped last value of each region). - scan_memory_for_exact_value NOT_EXACT_VALUE no longer yields every non-matching byte; aligned to target_value_size. - WindowsProcess default permission: PROCESS_VM_READ instead of PROCESS_ALL_ACCESS (least privilege). - Permission gate now requires VM_READ explicit OR all PROCESS_ALL_ACCESS bits set (was passing on any single bit). - ProcessOperationsEnum.PROCESS_TERMINATE was 0x0800 — same as PROCESS_SUSPEND_RESUME — making it a silent Enum alias. Corrected to 0x0001 per MSDN. - Linux MEMORY_BASIC_INFORMATION fields widened to 64-bit (regions > 4GB no longer truncate). - Linux /proc/<pid>/maps inode parsed as decimal (was hex). - Windows MEMORY_BASIC_INFORMATION layout picked per target via IsWow64Process (no more corruption when 64-bit Python attaches to 32-bit target). - macOS write_process_memory to a read-only page now transparently elevates protection via mach_vm_protect and restores it. - Linux scan filters shared mappings (parity with Win32/macOS). - read_process_memory(addr, str, n) now decodes with errors="replace" to match convert_from_byte_array. Performance - 6-8x speedup on numeric scans via struct.iter_unpack + inlined comparison loops per scan_type. NOT_EXACT_VALUE overlap check is now O(log m) via bisect_left. Tooling - CI: 3 OSes (ubuntu/windows/macos) x 6 Pythons (3.8-3.13), flake8 gate, mypy informational, pytest-cov. - Conventional Commits enforced on PR title (lint-pr-title.yml). - Dependabot config (open-pull-requests-limit: 0; security alerts still fire). - Auto-delete head branch on PR close. - Tk sample now requires Tk >= 8.6 and aborts with platform-specific install hints when missing or outdated. Removed - Unused PyMemoryEditor.linux.ptrace package. - Unused PyMemoryEditor.util.search (KMP/BMH never used in scan path). - Python 3.6 and 3.7 support (minimum is now 3.8).
The default permission on WindowsProcess is now PROCESS_VM_READ (a 2.0 breaking change), so existing write tests in test_editor.py started failing on Windows CI. Explicitly request VM_WRITE | VM_OPERATION on Win32; Linux and macOS ignore the kwarg as before.
- Add mypy override that ignores errors in win32/linux/macos backends. Each backend uses symbols (ctypes.windll, WINFUNCTYPE, Mach types) that only resolve on their target OS; mypy running on one host can't validate the others. - Cast() generic-return-vs-concrete-bytes/str in convert_from_byte_array. - Loosen get_c_type_of return type to Any (returns either _SimpleCData or ctypes.Array without a common base). - Tighten _as_bytes to only treat real bytes as a no-op (bytearray now goes through the bytes() conversion). - Move GetProcessIdByWindowTitle import into the function body so mypy on non-Windows hosts doesn't see it as undefined.
test_search_by_int and test_search_by_float iterate every address yielded by search_by_value_between and read it back to verify the value. A page mapped at scan time can be decommitted/protected before the subsequent read — the syscall now surfaces this as OSError (it used to silently return zeros). Wrap the read in try/except OSError, matching the pattern already used in test_search_by_string.
VirtualQueryEx requires PROCESS_QUERY_INFORMATION (or PROCESS_QUERY_ LIMITED_INFORMATION) in addition to PROCESS_VM_READ. With only PROCESS_VM_READ — the previous default — VirtualQueryEx returns 0, so get_memory_regions / snapshot_memory_regions / search_by_value* / search_by_addresses all came back empty. Exposed by Windows CI when test_region_snapshot opened a process without an explicit permission. The new default is PROCESS_VM_READ | PROCESS_QUERY_INFORMATION (exposed as the DEFAULT_PERMISSION constant). README and CHANGELOG updated. tests/test_editor.py also adds PROCESS_QUERY_INFORMATION to its permission combo to make the read-back loop in search-by-value tests robust across Windows versions (it happened to work in Python 3.11 by implicit grant, but the explicit bit makes it deterministic).
GitHub's macOS-latest runner pool is much smaller than ubuntu/windows. Running 6 Python versions × macos in parallel stalls the PR for 10+ minutes in queue without acquiring a runner. macOS only validates that the Mach backend works — the Python version doesn't change that surface, so we drop down to a single (stable) Python on macos-latest while keeping the full 6-version matrix on ubuntu/windows.
The macos-latest (Apple Silicon arm64) runner pool is heavily congested on free-tier accounts — jobs sit in queue for 15+ minutes without acquiring a runner. The macos-13 (Intel x86_64) pool is much larger and exercises identical code paths: Mach VM structs are fixed-size by design (mach_port_t = uint32, mach_vm_address_t = uint64, etc.), so x86_64 and arm64 hit the same struct layout and syscall surface.
GitHub-hosted macOS runner pools are often congested and a job can sit in queue for 30+ minutes without acquiring a runner. Stop the PR from stalling on this: - continue-on-error: true for macOS jobs (ubuntu and windows still gate the merge; macOS is best-effort) - 25-minute timeout caps the wait; real test runs finish in well under 10 min when a runner is available
timeout-minutes counts execution time, not queue time, so a macOS job stuck waiting for a runner can stall a PR indefinitely. The fix: - Split macOS into its own build-macos job - Gate it with: if: github.event_name != 'pull_request' - Runs on push-to-main, weekly cron, and workflow_dispatch — never blocks a PR PRs are gated by ubuntu × 5 Pythons + windows × 5 Pythons + lint, which already provides cross-platform coverage. The Mach backend is validated by the merged code via cron/dispatch and by local self-process tests in dev.
GitHub-hosted macOS runners are heavily congested on free-tier accounts. Even with continue-on-error and timeouts, the job blocked the workflow UI for tens of minutes per run. The Mach backend is covered by local self-process tests in dev; contributors with macOS hardware can run the suite directly.
Drop PyMemoryEditor.sample (Tk) in favour of PyMemoryEditor.app, a PySide6/Qt app that exercises every public surface of the library: all eight ScanTypesEnum modes, the five value types (bool/int/float/str/bytes), search_by_value / search_by_value_between (with progress_information, writeable_only, region snapshot reuse), search_by_addresses, read_process_memory, write_process_memory, get_memory_regions / snapshot_memory_regions, plus value freezing and a live hex viewer. Wired up as the pymemoryeditor CLI and a new [app] optional-dependencies extra for PySide6. Also extends the .flake8 ignore list with E203 (black-compatible slice spacing, mirroring the existing W503 ignore), and updates the README + CONTRIBUTING accordingly.
Black-style reformatting (slice spacing, line breaks around imports, blank line after class docstrings, etc.) across the cross-platform backends, util/, process/, the tests, and the two app modules the editor revisited after the previous commit. No behaviour changes; the .flake8 already tolerates E203 / W503 for compatibility with this style, so make lint stays green.
ctypes only declared restype for these syscalls; the default C-int width for argument coercion is narrower than the iovec pointer representation on some Linux builds (e.g. 32-bit hosts). The result is that pointers could be silently truncated before the kernel sees them — same class of bug the Win32 backend was audited for in v2, where every API now declares argtypes explicitly.
The Ubuntu runner ships without libEGL/libGL/libxkbcommon/libfontconfig or the XCB stack, so pytest-qt's pytest_configure hook crashed at `import QtGui` with `libEGL.so.1: cannot open shared object`. Install the required system packages on Linux and pin QT_QPA_PLATFORM=offscreen for the pytest step so Qt never tries to reach a display server.
JeanExtreme002
force-pushed
the
jeanextreme002/macos-support-and-hardening
branch
from
99fad24 to
37b9679
Compare
- macOS backend via Mach VM APIs (task_for_pid, mach_vm_read_overwrite, mach_vm_write, mach_vm_region, mach_vm_protect with restore + warning). - Strict partial-read/write checks on all three backends: Win32 ReadProcessMemory/WriteProcessMemory, Linux process_vm_readv/writev, and macOS mach_vm_read_overwrite now raise on short transfers instead of silently returning mixed real-bytes/zero buffers. - Win32: explicit argtypes/restype on every ctypes binding; use_last_error=True so GetLastError() surfaces; strict permission bitmask gate (replaces the loose PROCESS_ALL_ACCESS subset check); IsWow64Process dispatch picks the right MEMORY_BASIC_INFORMATION layout for 64-bit Python attached to a 32-bit target. - Shared layer under process/scanning.py + process/region.py owns chunking, boundary handling, gap detection, transient-error classification, and enrich_region predicates — ~350 LOC of per-backend duplication removed. - Fast numeric scan path (struct.iter_unpack + inlined per-scan_type loops, signed/IEEE-754 dispatch) ~6-8x faster than the prior version and now correct for signed ints and negative floats. - Replace the Tk demo with a Cheat-Engine-style Qt (PySide6) app exposed as the `pymemoryeditor` CLI (opt-in `[app]` extra). - Default Windows permission lowered to PROCESS_VM_READ | PROCESS_QUERY_INFORMATION (writers must opt in explicitly). PROCESS_TERMINATE corrected from 0x0800 (alias of PROCESS_SUSPEND_RESUME) to 0x0001 per MSDN. - snapshot_memory_regions() materializes + tags regions so refine workflows skip per-call enumeration and re-sort. - search_by_addresses yields (addr, None) for gap and out-of-bounds addresses instead of dropping them; NOT_EXACT_VALUE moved to bisect_left over match positions (O(n*m) -> O(n*log m)). - Linux: MEMORY_BASIC_INFORMATION widened to 64-bit, fixed-size inline byte arrays for Privileges/Path (avoids c_char_p lifetime UB), shared mappings skipped in scans, inode parsed as decimal (was hex). - LinuxProcess / MacProcess now warn (UserWarning) when `permission` is passed with a non-None value — the kwarg is accepted for cross-platform parity but had been silently discarded, hiding bugs in cross-platform callers that expected write access. - Hierarchy of typed exceptions under PyMemoryEditorError; py.typed marker; cross-platform AnyProcess alias under TYPE_CHECKING. - Python 3.8 dropped (EOL Oct 2024); psutil pinned >=5.9,<7. - New tests: scan/property-based (hypothesis), partial IO, str boundary, region snapshot, chunking integration, win32 permissions, macOS protect, process lookup, app smoke, cheat poll worker.
- CI matrix: drop 3.8/3.9, test 3.10–3.13 - pyproject: drop 3.9 classifier - README: update badges to reflect 3.10+ baseline - Remove stale Python 3.8 mentions from comments and CHANGELOG
- Process picker: switch from VMS (huge virtual ranges on macOS — every process looked like hundreds of GB) to phys_footprint via proc_pid_rusage, matching Activity Monitor's "Memory" column. Falls back to RSS for protected system processes where proc_pid_rusage returns EPERM. - Memory Map / Results "Open in Hex Viewer": signals were declared as Signal(int, ...), which marshals to C++ signed 32-bit and overflows for 64-bit addresses (common on macOS arm64 where ASLR puts mappings above 0x1_0000_0000). The overflow silently dropped the slot connection. Use qulonglong instead.
Ships an SVG icon under PyMemoryEditor/app/assets/ and wires it onto the QApplication, MainWindow and OpenProcessDialog. The icon is rasterized at runtime via QSvgRenderer at 8 sizes (16..512) so it stays crisp in taskbars, title bars and HiDPI alt-tab thumbnails without depending on Qt's SVG image plugin being registered.
Widen the outer and right splitters to 4px, add a hover color and small margins around the handles, and inset the surrounding panels so the dividers no longer collide at a "T" intersection or sit flush against toolbar/table edges.
Adds an "Edit Selected" button + context-menu action to the cheat table that opens a dialog to overwrite description, value type, and/or value on every targeted row (mouse-selected union with Active-checked rows). Each field has an Apply toggle so unchecked fields stay untouched, write failures are collected and surfaced in a single warning. Also rebrands the window titles, About box and README from "Qt app" to just "App".
Restructures the README around a quick-start → usage guide → platform notes flow, with collapsible per-OS sections and an app showcase. Removes the unused `docs` extra from pyproject.toml.
…ndows - Remove OpenProcess(window_title=...) from AbstractProcess and all platform subclasses; drop the supporting Win32 plumbing (GetProcessIdByWindowTitle, EnumWindows / GetWindowTextW / GetWindowThreadProcessId bindings, WindowNotFoundError, WNDENUMPROC) and stop exporting WindowNotFoundError. - WindowsProcess default permission goes back to read+write (VM_READ | VM_WRITE | VM_OPERATION | QUERY_INFORMATION). The v2.0 read-only default added friction for the common scan+poke workflow; callers who want a read-only handle pass the narrower mask explicitly. - Tidy a few app/ type annotations and drop "type: ignore" comments that no longer apply.
The Open Process dialog wraps its model in a QSortFilterProxyModel whose default lessThan compares Qt.DisplayRole strings, so NumericItem.__lt__ was never consulted and PID/Memory columns sorted lexicographically. Override lessThan to delegate to the source items.
…variants Swap the stock blue Fusion palette for a Kali-style terminal look: near-black graphite backgrounds, teal accent, and dedicated primary/secondary/danger QPushButton styles with hover/pressed/disabled states. Recolor the app icon and dialog hints to match, mark the scanner's Next Scan/Cancel buttons with the new object names, and install an app-wide event filter that gives every QPushButton a pointing-hand cursor (Qt QSS ignores the CSS cursor property).
Switch the Read, First Scan and Open Process buttons from the filled primary style to the lighter secondary outline, and lift the Cancel button's padding so it matches the secondary Open Process height.
…yberpunk palettes Extract the dark theme into a Theme dataclass and expose five presets via a toolbar menu. Selection is persisted across sessions through QSettings, defaulting to the existing Kali Teal palette.
PyPI renders the README in isolation and does not resolve relative paths the way GitHub does, so the logo and the new app screenshot would appear broken on the package page. Switch both <img> tags to raw.githubusercontent URLs on main.
Qt appends QGuiApplication.applicationDisplayName to window titles on Windows/Linux but not on macOS, so embedding "PyMemoryEditor App" in setWindowTitle produced a duplicated suffix off-mac.
JeanExtreme002
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Major v2.0 release with three pillars:
OpenProcess(...)surface as Windows/Linux.argtypes/restypeon every ctypes binding, partial-read/write detection, WOW64-aware MBI layout, shared layer for chunking and region enrichment.sample, shipped as thepymemoryeditorCLI under the optional[app]extra.Full history is in
git log master..HEAD; this description hits the load-bearing points.Library
macOS backend
task_for_pid,mach_vm_read_overwrite,mach_vm_write,mach_vm_region,mach_vm_protect(with restore +UserWarningon protection flips).com.apple.security.cs.debuggeror SIP off + root.phys_footprintviaproc_pid_rusage(matches Activity Monitor), falling back to RSS for protected system processes.Hardening (Windows + Linux + macOS)
ReadProcessMemory/WriteProcessMemory,process_vm_readv/writev, andmach_vm_read_overwritenow raise on short transfers instead of silently returning zero-padded buffers.argtypes/restypeon every binding,use_last_error=True, strict permission bitmask gate,IsWow64Processdispatch for the correctMEMORY_BASIC_INFORMATIONlayout when 64-bit Python attaches to a 32-bit target.MEMORY_BASIC_INFORMATIONwidened to 64-bit, inline byte arrays for Privileges/Path (no morec_char_plifetime UB), shared mappings skipped in scans,/proc/<pid>/mapsinode parsed as decimal.PROCESS_TERMINATEcorrected from0x0800(alias ofPROCESS_SUSPEND_RESUME) to0x0001.process/scanning.py+process/region.pyconsolidates chunking, gap detection, transient-error classification, andenrich_regionpredicates — ~350 LOC of per-backend duplication removed.Performance & API
struct.iter_unpack+ inlined per-ScanTypesEnumloops.NOT_EXACT_VALUEoverlap check nowO(n·log m)viabisect_left(wasO(n·m)).snapshot_memory_regions()+memory_regions=kwarg onsearch_by_value*/search_by_addressesfor refine-scan workflows.bufflengthoptional for numeric types (int→4,float→8,bool→1).PyMemoryEditorErrorhierarchy,AmbiguousProcessNameError,py.typedmarker,AnyProcessalias underTYPE_CHECKING.App (
PyMemoryEditor.app)Drop-in replacement for the old Tk
sample. PySide6/Qt, exposed as thepymemoryeditorCLI behind an opt-in[app]extra.ScanTypesEnummodes, the 5 value types (bool/int/float/str/bytes),search_by_value/search_by_value_between(withprogress_information,writeable_only, snapshot reuse),search_by_addresses,read/write_process_memory,get_memory_regions/snapshot_memory_regions, value freezing, live hex viewer.QSettings.lessThanon the proxy model).phys_footprint; 64-bit-safequlonglongsignals so addresses above0x1_0000_0000don't overflow the slot dispatch.Breaking changes
OpenProcess(window_title=...)removed (and the supporting Win32 plumbing +WindowNotFoundErrorexport).ProcessOperationsEnum.PROCESS_TERMINATEvalue changed from0x0800→0x0001. Callers relying on the buggy alias were actually terminating processes when they asked to suspend/resume.PyMemoryEditor.sample(Tk) removed in favor ofPyMemoryEditor.app(Qt, opt-in via[app]).PyMemoryEditor.linux.ptraceandPyMemoryEditor.util.search(unused KMP/BMH) removed.LinuxProcess/MacProcessnow emitUserWarningwhenpermission=is passed with a non-Nonevalue (was silently discarded before).Tooling
push/cron/workflow_dispatch(macOS runner pool congestion was stalling PRs).flake8as a gate;mypyinformational with per-backend overrides;pytest-covreporting;pytest-qtwithQT_QPA_PLATFORM=offscreenand the Linux Qt system libs installed in CI.lint-pr-title.yml).open-pull-requests-limit: 0— security alerts still fire).make install-app/make run-apptargets.Test plan
flake8 PyMemoryEditor tests— cleantests/test_macos_protect.py(read-only-page write throughmmap+mprotect) and local self-process runsIsWow64ProcesstestAmbiguousProcessNameErrorandcase_sensitivecovered bypsutil-mocked teststests/test_app_smoke.py) andcheat_poll_workerunit testspymemoryeditorapp on each OS