- 09 Oct, 2024 2 commits
-
-
Xiaomeng Zhao authored
-
Xiaomeng Zhao authored
-
- 08 Oct, 2024 20 commits
-
-
Xiaomeng Zhao authored
docs: update CUDA acceleration guides and README content
-
myhloli authored
- Update GPU hardware support information in README.md and README_zh-CN.md - Enhance CUDA acceleration guides for Ubuntu and Windows - Modify README_zh-CN.md to reflect changes in GPU requirements and configurations - Update TODO list to mark semantic reading order as completed
-
myhloli authored
- Update GPU hardware support information in README.md and README_zh-CN.md - Enhance CUDA acceleration guides for Ubuntu and Windows - Modify README_zh-CN.md to reflect changes in GPU requirements and configurations - Update TODO list to mark semantic reading order as completed
-
Xiaomeng Zhao authored
docs: add filename to wget command in model download scripts
-
myhloli authored
- Update wget commands in both English and Chinese documentation to specify the filename - Improve clarity and prevent potential filename conflicts when downloading the scripts
-
Xiaomeng Zhao authored
feat(docs): automate model download and configuration
-
myhloli authored
- Add scripts to download models and update configuration file - Remove manual steps for modifying model paths - Update documentation for both ModelScope and HuggingFace model downloads - Improve user experience by automating the entire process
-
myhloli authored
- Add scripts to download models and update configuration file - Remove manual steps for modifying model paths - Update documentation for both ModelScope and HuggingFace model downloads - Improve user experience by automating the entire process
-
Xiaomeng Zhao authored
feat(layoutreader): support local model directory and improve model loading
-
myhloli authored
Added a link to the layoutreader repository in the Related Projects sections of both the README.md and README_zh-CN.md files. This addition helps to provide users with more resources and tools related to document layout analysis and processing.
-
myhloli authored
docs: update model download instructions for version 0.9.x and later- Add note about separate download for layoutreader model in version 0.9.x and later - Include example code for downloading layoutreader model using ModelScope - Clarify that previous download methods do not support updating to version 0.9.x and later
-
myhloli authored
- Add function to get local LayoutReader model directory- Check and use local model directory if available - Fall back to online model if local directory not found - Update model initialization to support local path - Refactor model loading in singleton class
-
Xiaomeng Zhao authored
fix: caption|footnote match algorithm
-
icecraft authored
-
Xiaomeng Zhao authored
fix: caption or footnote match algorithm
-
icecraft authored
-
Xiaomeng Zhao authored
perf(pdf_extract_kit): conditional memory cleanup based on GPU capacity
-
myhloli authored
- Introduce a conditional memory cleanup step in the PDF extraction process - Assess available GPU memory before deciding to perform memory cleanup- Log the time taken for garbage collection when it occurs - This optimization helps to balance performance and resource utilization
-
Xiaomeng Zhao authored
feat: add arXiv paper link to header and adjust PDF parsing logic
-
myhloli authored
feat: add arXiv paper link to header and adjust PDF parsing logic- Add arXiv paper link to the header template for easy access to the latest research paper. - Modify the PDF parsing logic to handle edge cases more accurately, particularly in determining the number of lines in a block based on its height.
-
- 06 Oct, 2024 2 commits
-
-
Xiaomeng Zhao authored
refactor(model): improve timing information and performance
-
myhloli authored
- Enhance timing output precision to two decimal places for better readability- Calculate and log document analysis speed in pages per second - Optimize logging for YOLO and table recognition processes - Remove unnecessary comments and improve code efficiency
-
- 30 Sep, 2024 6 commits
- 29 Sep, 2024 2 commits
-
-
myhloli authored
- Insert lines into blocks based on median line height- Calculate block index using line indices median - Remove virtual line information for table and image blocks - Enhance line sorting algorithm for different block types - Add line height calculation function
-
myhloli authored
The clean_memory function has been removed from pdf_parse_union_core_v2.py due to it not being used. This change streamlines the code and prevents potential confusion regarding its purpose.
-
- 28 Sep, 2024 3 commits
-
-
myhloli authored
Update import statements in `pdf_parse_union_core_v2.py` to directly import `prepare_inputs`, `boxes2inputs`, and `parse_logits` from `magic_pdf.model.v3.helpers` instead of from `magic_pdf.model.v3`. This change streamlines the imports, making the code more readable and maintaining a cleaner approach to modular design.
-
myhloli authored
Adapt import statements in `pdf_parse_union_core_v2.py` to reflect the updated packagestructure, changing from the `magic_pdf.v3.helpers` module to the `magic_pdf.model.v3` module. This ensures compatibility with the revised directory layout.
-
myhloli authored
Blocks without lines are now correctly indexed even when they contain textual content rendered as images. The sorting logic has been updated to accommodate this scenario. Additionally, the LayoutLMv3 model initialization has been enhanced to utilize bfloat16 precision on devices that support it, offering potential performance benefits on supported hardware.
-
- 27 Sep, 2024 5 commits
-
-
myhloli authored
Removed redundant sorting of lines by model and optimized calculation of block indexes by using a single pass through the sorted lines. This change simplifies the code and potentially improves performance by reducing the number of sortingoperations and unnecessary iterations over blocks without lines.
-
myhloli authored
refactor(draw_bbox): remove commented-out code and streamline bbox drawingRemoved legacy commented-out code related to layout_bbox_list from draw_bbox.py, which was used for diagnostic purposes and was no longer necessary. This change streamlines the codebase and clarifies the drawing process of bounding boxes on PDF pages. The update also adjusts the order of operations slightly for improved readability without altering the functionality.
-
myhloli authored
Introduce torch and transformers libraries to support new ML features.Ensure version compatibility by adding torch version within the range 2.2.2 to 2.3.1and include the necessary transformers library.
-
myhloli authored
refactor(pdf_parse_union_core_v2): implement model initialization within classRefactored model initialization to be handled by a singleton class to ensure that model instances are reused across calls, avoiding redundant initializations. Removed logger information that was commented out and ensured consistency in logging behavior.
-
myhloli authored
Refactor the draw bbox functions by removing unused imports and simplifying the code logic for drawing layout and line sorting bounding boxes. Adjust the debug configuration to enable content list dumping and disable markdown making mode.
-