From 04973cf50d2089dca127955a205a691d2619e39b Mon Sep 17 00:00:00 2001 From: Sean Harding Date: Tue, 11 Aug 2026 19:02:52 -0500 Subject: [PATCH 1/2] Name the reason a Nikon High Efficiency NEF will not open Reported as "NegPy won't open this NEF" on a Z 8 file. It is not a NEF fault: the camera recorded it in High Efficiency (HE / HE*), which is intoPIX TicoRAW under a licensed codec. Nothing in the tags gives it away. The TIFF Compression reads 34713, "Nikon NEF Compressed" -- the same value a lossless NEF carries -- so libraw parses the file, reports the full 8280x5520 sensor and a valid crop, and only fails when the payload is unpacked, with "Unsupported file format or not RAW file". To someone whose other NEFs all work, that reads as a corrupt file. Only the payload distinguishes them: the strip opens with a plain-text intoPIX vendor marker, and the ratio is 0.357 against ~0.55-0.70 for lossless. `unsupported_raw_reason` recognises that marker and returns wording that says what to do -- re-shoot Lossless Compressed, or convert to DNG. The loader stays lazy: the check runs only once a decode has already failed, so a file that works never pays for it (measured 1.5 ms on a lossless NEF, which is 1.5 ms saved on every decode of every good file). Anything it does not recognise keeps libraw's own message, which is the honest answer for a genuinely corrupt or unknown file. Ruled out on the way: stripped MakerNotes (intact, 67 KB across 77 Nikon tags) and a truncated file (every other IFD reads, including a full-resolution JPEG preview). Verified through the app headless -- the status line, the error signal and load_failed all carry the new wording -- and the detector is tested against a byte-built synthetic HE NEF as well as the real one, so it does not depend on a single sample. --- docs/CHANGELOG.md | 1 + docs/USER_GUIDE.md | 2 + negpy/desktop/workers/render.py | 10 ++- negpy/infrastructure/loaders/helpers.py | 44 +++++++++++++ tests/test_unsupported_raw_reason.py | 84 +++++++++++++++++++++++++ 5 files changed, 139 insertions(+), 2 deletions(-) create mode 100644 tests/test_unsupported_raw_reason.py diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 99c0a51b..9fe9c3e9 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- Fix: **A Nikon High Efficiency NEF now says why it cannot be opened.** The Z 8 and Z 9 can record raw in HE or HE*, which is intoPIX TicoRAW under a licensed codec. Such a file keeps the `.NEF` extension and the same TIFF compression tag as an ordinary lossless NEF, so it parses cleanly and reports full sensor dimensions, and only fails when the pixels are unpacked — as "Unsupported file format or not RAW file", which reads as a corrupt file to someone whose other NEFs all work. NegPy now recognises the payload and says what it is and what to do: re-shoot in Lossless Compressed, or convert to DNG. Decoding these is not possible without the codec. The check runs only after a decode has already failed, so files that work pay nothing for it. - Fix: **The session panel's toolbar measures its own contents correctly.** It sized every item by `sizeHint()`, which is only a preference — `setFixedWidth` does not change it, and a bare `QFrame` has none at all and reports -1. So the 1 px dividers were each given a 3 px slot that Qt then clamped the ink back into, leaving 2 px of dead space beside them and pushing buttons into the **»** menu sooner than the width required. The bar also reserved room for the **»** button before knowing one was needed, which on a bar ending in a divider evicted a real button and so conjured the very chevron it was reserving for. Both now measure what will actually be laid out. This is what `tests/test_overflow_bar.py::test_overflow_button_stays_hidden_when_only_a_separator_would_spill` has been failing on since #701 — on macOS, where a QToolButton's natural hint is wider than the width it is fixed to; CI runs Linux, where it is not. - Fix: **Dialog buttons on macOS no longer underline a letter that does nothing.** Qt puts the mnemonic in its own standard-button text, so a Yes/No box asks for `&Yes`, and the Fusion style NegPy uses for the dark theme draws the underline. macOS has no mnemonic convention, though, so Qt binds no key there — the native style answers that hint the other way and draws nothing. The underline is now suppressed on macOS only; Alt+letter still works, and is still shown, on Windows and Linux. - Fix: **The filmstrip no longer shows one frame's picture on another frame's thumbnail.** A render carries the frame it was started for, but the finished pixels were filed under whichever frame happened to be selected when they arrived — so clicking to the next frame before a full-res decode finished cached the previous frame's image on the new one, where it stayed until that cell was clicked and re-rendered. The thumbnail now follows the render's own frame, as the render memo already did. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index f7fa41f3..89ea2737 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -72,6 +72,8 @@ NegPy reads the tree straight from disk and never creates, renames, moves or del ### Importing & managing files +**A note on Nikon High Efficiency raw.** The Z 8 and Z 9 can record NEFs in **High Efficiency (HE)** or **HE\***, which use a licensed codec NegPy cannot decode. Such a file is still called `.NEF` and still carries the same TIFF compression tag as an ordinary lossless NEF, so nothing about it looks unusual until it fails to open — NegPy names the reason rather than reporting a generic unsupported-file error. Re-shoot in **Lossless Compressed** NEF, or convert with Adobe DNG Converter. Lossless NEFs from the same cameras open normally. + Toolbar buttons, left to right: * **Add files** / **Add folder**: load individual images or every image in a folder. Pick a folder that only holds *other* folders and NegPy reveals it in the Library section instead of reporting that it found nothing. Dropping a folder on the window does the same. diff --git a/negpy/desktop/workers/render.py b/negpy/desktop/workers/render.py index 927aec95..c1f2189d 100644 --- a/negpy/desktop/workers/render.py +++ b/negpy/desktop/workers/render.py @@ -13,6 +13,7 @@ from negpy.features.hdr.models import HdrConfig, hdr_active from negpy.features.geometry.batch_autocrop import detect_crop_candidate, resolve_roll_crops from negpy.features.process.sensor import apply_sensor_correction, effective_sensor_matrix +from negpy.infrastructure.loaders.helpers import unsupported_raw_reason from negpy.features.rgbscan.models import RgbScanConfig, is_rgb_triplet from negpy.features.stitch.models import StitchConfig, stitch_active from negpy.features.geometry.processor import GeometryProcessor @@ -736,8 +737,13 @@ def process(self, task: PreviewLoadTask) -> None: ) except Exception as e: logger.exception(f"Asset load failed: {task.file_path}") - self.error.emit(str(e)) - self.load_failed.emit(task.file_path, str(e)) + # libraw reports "Unsupported file format or not RAW file" for a file whose + # tags it parsed perfectly and whose payload it cannot decode, which reads as + # "your NEF is broken". Ask why only once the decode has actually failed, so + # the check costs nothing on the files that work. + message = unsupported_raw_reason(task.file_path) or str(e) + self.error.emit(message) + self.load_failed.emit(task.file_path, message) def _detect_mode(self, task: PreviewLoadTask, raw) -> str: """Classify film process mode; re-decode no-WB since the C41 mask is hidden by camera WB.""" diff --git a/negpy/infrastructure/loaders/helpers.py b/negpy/infrastructure/loaders/helpers.py index f59989d1..2c444600 100644 --- a/negpy/infrastructure/loaders/helpers.py +++ b/negpy/infrastructure/loaders/helpers.py @@ -1,3 +1,4 @@ +import os import io from types import SimpleNamespace from typing import Any, Optional, Tuple @@ -228,6 +229,49 @@ def camera_wb_multipliers(raw: Any) -> Optional[list]: return wb +#: Nikon's High Efficiency (HE / HE*) raw on the Z 8 / Z 9 is intoPIX TicoRAW carrying a +#: plain-text vendor marker at the head of the strip. The TIFF tag still reads 34713 +#: ("Nikon NEF Compressed"), the same value a lossless NEF uses, so tags cannot tell them +#: apart -- only the payload can. +_TICORAW_MARKER = b"INTOPIX" +_NEF_COMPRESSED = 34713 + + +def unsupported_raw_reason(file_path: str) -> Optional[str]: + """Why libraw cannot decode this raw, in words a photographer can act on. + + None when nothing recognised is wrong -- the caller then reports libraw's own error, + which is right for a genuinely corrupt or unknown file. This exists because the useful + cases are indistinguishable from corruption by their tags: a High Efficiency NEF parses + perfectly, reports full sensor dimensions, and only fails when the payload is unpacked. + """ + if os.path.splitext(file_path)[1].lower() != ".nef": + return None + try: + import tifffile + + with tifffile.TiffFile(file_path) as tif: + for sub in tif.pages[0].pages or []: + tags = getattr(sub, "tags", None) + compression = tags.get("Compression") if tags else None + if compression is None or int(compression.value) != _NEF_COMPRESSED: + continue + offsets = tags.get("StripOffsets") + if offsets is None: + continue + offset = offsets.value[0] if isinstance(offsets.value, (tuple, list)) else int(offsets.value) + with open(file_path, "rb") as f: + f.seek(int(offset)) + if _TICORAW_MARKER in f.read(64): + return ( + "This NEF uses Nikon High Efficiency (HE / HE*) compression, which NegPy cannot decode. " + "Re-shoot in Lossless Compressed NEF, or convert the file to DNG with Adobe DNG Converter." + ) + except Exception: + return None + return None + + def get_supported_raw_wildcards() -> str: """ Returns raw formats as string for file dialogs. diff --git a/tests/test_unsupported_raw_reason.py b/tests/test_unsupported_raw_reason.py new file mode 100644 index 00000000..afa16372 --- /dev/null +++ b/tests/test_unsupported_raw_reason.py @@ -0,0 +1,84 @@ +"""Explain a raw NegPy cannot decode, when the reason is knowable. + +Nikon's High Efficiency raw (Z 8 / Z 9) is intoPIX TicoRAW, but its TIFF Compression tag +reads 34713 — the same value a lossless NEF uses. So the tags say "ordinary NEF", libraw +parses the file and reports full sensor dimensions, and the failure only arrives when the +payload is unpacked, as "Unsupported file format or not RAW file". That reads as "your file +is corrupt" to someone whose other NEFs all work. +""" + +import struct + +import pytest + +from negpy.infrastructure.loaders.helpers import unsupported_raw_reason + + +def _nef(tmp_path, name, strip_payload: bytes, compression: int = 34713): + """A minimal little-endian TIFF with one SubIFD carrying `strip_payload`. + + Only the parts the detector reads: IFD0 with a SubIFDs tag, and a SubIFD with + Compression and StripOffsets. + """ + path = tmp_path / name + strip_offset = 512 + sub_offset = 200 + ifd0_offset = 8 + + def ifd(entries, next_ifd=0): + out = struct.pack(" Date: Tue, 11 Aug 2026 19:11:57 -0500 Subject: [PATCH 2/2] Wrap a long status toast instead of running it off the canvas The toast was an unwrapped QLabel. Fine for "merging exposures"; not for a message that has to say what went wrong and what to do about it, which ran off both edges. It now wraps within a share of the canvas width. Two bounds, because one is not enough: the maximum keeps it clear of the edges, and the minimum stops Qt's wrapping sizeHint -- which aims for a squarish block -- folding a sentence into five narrow lines when it has the width for two. Short toasts keep their natural size. The bounds are applied when the message is posted as well as on resize, since a load failure can post one before the HUD has ever been resized. Measured at 1400px canvas: the unsupported-raw message goes from one overflowing line to 770x52, and "merging exposures" is unchanged at 327x34. The NEF message is shortened to suit, saying the same thing in half the words. Noted while rendering it, not changed: showMessage lowercases every toast, so this one reads "nikon high efficiency (he) raw -- negpy cannot decode this format ... convert to dng". That is a good voice for activity toasts and it flattens the acronyms in anything the user has to act on. Left alone because it is a house-style decision rather than a fault, and it predates this change; worth a look if error toasts become common. --- docs/CHANGELOG.md | 1 + negpy/desktop/view/canvas/hud.py | 21 ++++++++ negpy/infrastructure/loaders/helpers.py | 4 +- tests/test_hud_toast.py | 66 +++++++++++++++++++++++++ 4 files changed, 90 insertions(+), 2 deletions(-) create mode 100644 tests/test_hud_toast.py diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 9fe9c3e9..62692a7a 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -2,6 +2,7 @@ ## Unreleased +- Fix: **A long status message now wraps instead of running off the canvas.** The toast was a single unwrapped line, which was fine for "merging exposures" but cut off anything that had to explain itself — such as why a file will not open. It now wraps within a share of the canvas width, using the room it has before folding; short toasts are unchanged. - Fix: **A Nikon High Efficiency NEF now says why it cannot be opened.** The Z 8 and Z 9 can record raw in HE or HE*, which is intoPIX TicoRAW under a licensed codec. Such a file keeps the `.NEF` extension and the same TIFF compression tag as an ordinary lossless NEF, so it parses cleanly and reports full sensor dimensions, and only fails when the pixels are unpacked — as "Unsupported file format or not RAW file", which reads as a corrupt file to someone whose other NEFs all work. NegPy now recognises the payload and says what it is and what to do: re-shoot in Lossless Compressed, or convert to DNG. Decoding these is not possible without the codec. The check runs only after a decode has already failed, so files that work pay nothing for it. - Fix: **The session panel's toolbar measures its own contents correctly.** It sized every item by `sizeHint()`, which is only a preference — `setFixedWidth` does not change it, and a bare `QFrame` has none at all and reports -1. So the 1 px dividers were each given a 3 px slot that Qt then clamped the ink back into, leaving 2 px of dead space beside them and pushing buttons into the **»** menu sooner than the width required. The bar also reserved room for the **»** button before knowing one was needed, which on a bar ending in a divider evicted a real button and so conjured the very chevron it was reserving for. Both now measure what will actually be laid out. This is what `tests/test_overflow_bar.py::test_overflow_button_stays_hidden_when_only_a_separator_would_spill` has been failing on since #701 — on macOS, where a QToolButton's natural hint is wider than the width it is fixed to; CI runs Linux, where it is not. - Fix: **Dialog buttons on macOS no longer underline a letter that does nothing.** Qt puts the mnemonic in its own standard-button text, so a Yes/No box asks for `&Yes`, and the Fusion style NegPy uses for the dark theme draws the underline. macOS has no mnemonic convention, though, so Qt binds no key there — the native style answers that hint the other way and draws nothing. The underline is now suppressed on macOS only; Alt+letter still works, and is still shown, on Windows and Linux. diff --git a/negpy/desktop/view/canvas/hud.py b/negpy/desktop/view/canvas/hud.py index de739c32..8a5ceb9b 100644 --- a/negpy/desktop/view/canvas/hud.py +++ b/negpy/desktop/view/canvas/hud.py @@ -4,6 +4,12 @@ from negpy.desktop.view.styles.theme import THEME _DEFAULT_TOAST_MS = 2500 +#: Toast wrapping bounds. The ratio keeps a long message clear of the corner pills; the +#: floor stops it collapsing to a sliver on a narrow canvas. +_TOAST_WIDTH_RATIO = 0.55 +_TOAST_MIN_WIDTH = 320 +#: Horizontal padding in _TOAST_QSS, so a one-line measurement matches the rendered pill. +_TOAST_PADDING = 40 _PILL_QSS = ( f"color: {THEME.text_secondary}; font-size: {THEME.font_size_xs}px; font-weight: 500; " @@ -46,6 +52,10 @@ def __init__(self, parent=None): lbl.setStyleSheet(_PILL_QSS) lbl.hide() self.toast.setStyleSheet(_TOAST_QSS) + # Wraps rather than running off the canvas: a message that has to explain itself — + # why a file will not open, and what to do about it — does not fit one line. + self.toast.setWordWrap(True) + self.toast.setAlignment(Qt.AlignmentFlag.AlignCenter) self.toast.hide() top = Qt.AlignmentFlag.AlignTop @@ -74,6 +84,9 @@ def __init__(self, parent=None): def resizeEvent(self, event) -> None: super().resizeEvent(event) self.progress.setGeometry(0, 0, self.width(), 3) + # A cap, not a width: wordWrap only wraps at the widget's width, and without a + # bound the label simply grows to the canvas. Short toasts still size to content. + self.toast.setMaximumWidth(max(_TOAST_MIN_WIDTH, int(self.width() * _TOAST_WIDTH_RATIO))) @staticmethod def _set_pill(lbl: QLabel, text: str) -> None: @@ -93,6 +106,14 @@ def showMessage(self, text: str, timeout: int = 0) -> None: self._toast_timer.stop() self.toast.hide() return + # Cap here as well as on resize: a toast can be posted before the HUD has ever been + # resized. And set the minimum too — a wrapping QLabel's sizeHint aims for a squarish + # block, so a long message folds into five narrow lines rather than using the width + # it is allowed. Short toasts keep their natural width. + cap = max(_TOAST_MIN_WIDTH, int(self.width() * _TOAST_WIDTH_RATIO)) + one_line = self.toast.fontMetrics().horizontalAdvance(text) + _TOAST_PADDING + self.toast.setMaximumWidth(cap) + self.toast.setMinimumWidth(min(cap, one_line)) self.toast.setText(text.lower()) self.toast.show() self._toast_timer.start(timeout if timeout > 0 else _DEFAULT_TOAST_MS) diff --git a/negpy/infrastructure/loaders/helpers.py b/negpy/infrastructure/loaders/helpers.py index 2c444600..01b0c2e7 100644 --- a/negpy/infrastructure/loaders/helpers.py +++ b/negpy/infrastructure/loaders/helpers.py @@ -264,8 +264,8 @@ def unsupported_raw_reason(file_path: str) -> Optional[str]: f.seek(int(offset)) if _TICORAW_MARKER in f.read(64): return ( - "This NEF uses Nikon High Efficiency (HE / HE*) compression, which NegPy cannot decode. " - "Re-shoot in Lossless Compressed NEF, or convert the file to DNG with Adobe DNG Converter." + "Nikon High Efficiency (HE) raw — NegPy cannot decode this format. " + "Re-shoot as Lossless Compressed, or convert to DNG." ) except Exception: return None diff --git a/tests/test_hud_toast.py b/tests/test_hud_toast.py new file mode 100644 index 00000000..d282e89f --- /dev/null +++ b/tests/test_hud_toast.py @@ -0,0 +1,66 @@ +"""A toast that has to explain something must stay on the canvas. + +Most toasts are a few words ("merging exposures"), but a failure has to say what went wrong +and what to do about it. Unwrapped, such a message ran off both edges of the canvas. +""" + +from PyQt6.QtWidgets import QWidget + +from negpy.desktop.view.canvas.hud import _TOAST_WIDTH_RATIO, CanvasHud + +LONG = "Nikon High Efficiency (HE) raw — NegPy cannot decode this format. Re-shoot as Lossless Compressed, or convert to DNG." +SHORT = "merging exposures" + + +def _hud(qapp, width=1400, height=900): + """The parent is returned too: dropping it lets Qt delete the HUD as its child.""" + parent = QWidget() + parent.resize(width, height) + hud = CanvasHud(parent) + hud.resize(width, height) + hud._keepalive = parent + return hud + + +def test_a_long_message_stays_within_the_canvas(qapp): + hud = _hud(qapp) + hud.showMessage(LONG, 5000) + hud.toast.adjustSize() + assert hud.toast.width() <= hud.width(), "the toast ran off the canvas" + assert hud.toast.width() <= int(hud.width() * _TOAST_WIDTH_RATIO) + 1, "it should leave room around itself" + + +def test_a_long_message_wraps_rather_than_truncating(qapp): + """Wrapping, not eliding — the remedy is in the second half of the sentence.""" + hud = _hud(qapp) + hud.showMessage(LONG, 5000) + hud.toast.adjustSize() + assert hud.toast.height() > hud.toast.fontMetrics().lineSpacing(), "expected more than one line" + assert hud.toast.text().replace("\n", " ").strip() == LONG.lower(), "no text may be dropped" + + +def test_a_short_message_keeps_its_natural_width(qapp): + """The floor must not inflate an ordinary toast into a banner.""" + hud = _hud(qapp) + hud.showMessage(SHORT, 3000) + hud.toast.adjustSize() + assert hud.toast.width() < int(hud.width() * _TOAST_WIDTH_RATIO), "a short toast should not fill the cap" + assert hud.toast.height() <= hud.toast.fontMetrics().lineSpacing() * 2 + + +def test_it_fits_before_the_hud_has_ever_been_resized(qapp): + """A load failure can post a toast during startup, before any resize event.""" + parent = QWidget() + parent.resize(1200, 800) + hud = CanvasHud(parent) + hud._keepalive = parent + hud.showMessage(LONG, 5000) + hud.toast.adjustSize() + assert hud.toast.width() <= max(1200, hud.width()) + + +def test_a_narrow_canvas_still_gets_a_usable_width(qapp): + hud = _hud(qapp, width=420) + hud.showMessage(LONG, 5000) + hud.toast.adjustSize() + assert hud.toast.width() >= 320, "the floor keeps it from collapsing to a sliver"