Add instant image translation (Google Lens style)#293
Open
quangdat99x wants to merge 1 commit into
Open
Conversation
New Alt+D hotkey: select a screen region, then OCR it, auto-detect the source language, translate, and paint the translation over the original text positions in a floating overlay — with inline source/target language dropdowns, copy, and Esc-to-close. - Positional Windows OCR with per-line boxes + multi-pack language auto-detect scored by writing-system match (UWP OCR has no confidence API): Translumo.OCR/WindowsOCR/WindowsOcrPositional.cs - AutoSourceGoogleTranslator (sl=auto) reusing the proxy-aware GoogleContainer, without touching the config-bound GoogleTranslator - ImageTranslationService orchestrates OCR + bounded-concurrency per-line translation - ImageTranslationOverlayWindow renders the frozen region snapshot with translated boxes and the toolbar; live re-translate on dropdown change - Hotkey/DI wiring + localization (en/ru/zh); default target Vietnamese Verified end to end: auto-detected en-US, translated to Vietnamese, and live-switched target to Spanish over a real screen region. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new Alt+D hotkey that turns a screen selection into a Google Lens style translation: OCR the region, auto-detect the source language, translate, and paint the translation over the original text positions in a floating overlay. A small toolbar offers source-override + target-language dropdowns (live re-translate), copy-all, and Esc/click-away to close. Default target is Vietnamese.
What''s inside
Translumo.OCR/WindowsOCR/WindowsOcrPositional.cs: keeps per-line bounding boxes and detects the source language by running the installed recognizer packs and scoring each result by how many characters match that language''s writing system (the UWP OCR API exposes no per-word confidence, so a script-match heuristic is used).AutoSourceGoogleTranslatorsendssl=auto, reusing the proxy-awareGoogleContainerand result regex without disturbing the config-boundGoogleTranslatorused by continuous translation.ImageTranslationServiceruns OCR then per-line translation with bounded concurrency.ImageTranslationOverlayWindowshows the frozen region snapshot with translated boxes at the original positions and the toolbar; changing a dropdown re-runs the pipeline and repaints.ImageTranslateKey(default Alt+D) acrossHotKeysConfiguration/HotKeysServiceManager, a configurable row in Hotkeys settings,ChatWindowViewModelhandler (region capture + overlay), DI registration, and localization strings (en/ru/zh).Notes / limitations
sl=auto.Test plan
dotnet build Translumo.sln -c Release— 0 errors.en-US, translated to Vietnamese over the original lines, and switching the target dropdown to Spanish live re-translated all lines. Copy and Esc work.🤖 Generated with Claude Code