61
Ideas/Suggestions / optimize PDFs for machine learning / AI model training
« Last post by edgaughan@hotmail.com on February 24, 2024, 11:58:19 PM »My company trains and grounds Large Language Model (LLM) with PDF files. The problem is the valuable part of a
PDF is the body text, while the Table of contents, footnotes, index, and headers/footers create problems (especially with semantic search).
Do any of your utilities allow for batch processing of files that will:
- delete all text below a point size (ie delete text =<9 points will remove foot notes and index)
- remove Table of Contents
- remove all text in margins
There is a lot of demand for a user-friendly tool that preps PDFs for machine learning.
PDF is the body text, while the Table of contents, footnotes, index, and headers/footers create problems (especially with semantic search).
Do any of your utilities allow for batch processing of files that will:
- delete all text below a point size (ie delete text =<9 points will remove foot notes and index)
- remove Table of Contents
- remove all text in margins
There is a lot of demand for a user-friendly tool that preps PDFs for machine learning.
Recent Posts