[MS] Add OCR layer service for embedded images and PDF scans#1541
Open
lesyk wants to merge 4 commits intomicrosoft:mainfrom
Open
[MS] Add OCR layer service for embedded images and PDF scans#1541lesyk wants to merge 4 commits intomicrosoft:mainfrom
lesyk wants to merge 4 commits intomicrosoft:mainfrom
Conversation
- Created HTML file with multiple images for testing OCR extraction. - Added several PDF files with different layouts and image placements to validate OCR functionality. - Introduced PPTX files with complex layouts and images at various positions for comprehensive testing. - Included XLSX files with multiple images and complex layouts to ensure accurate OCR extraction. - Implemented a new test suite in `test_ocr.py` to validate OCR functionality across all document types, ensuring context preservation and accuracy.
- Refactor image extraction and processing in PDF, PPTX, and XLSX converters for improved readability and consistency. - Implement detailed validation for OCR text positioning relative to surrounding text in test cases. - Introduce comprehensive tests for expected OCR results across various document types, ensuring no base64 images are present. - Improve error handling and logging for better debugging during OCR extraction.
zashed
approved these changes
Jan 27, 2026
zashed
approved these changes
Jan 27, 2026
lesyk
changed the title
lesyk
changed the title
lesyk
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces comprehensive OCR support to MarkItDown, enabling extraction of text from images embedded in documents (PDF, DOCX, XLSX, PPTX) using multiple backends (Tesseract, EasyOCR, LLM Vision, Azure Document Intelligence). It adds a unified OCR service layer with graceful fallback between backends, automatic detection and handling of scanned PDFs, and inline extraction of image text in DOCX files. The README is updated with detailed usage examples for the new OCR features.
#1344