RAG for PDFs: how to parse, clean, and index complex documents

To build an effective RAG system over PDF documents, you must first understand a harsh truth: the PDF format was invented in 1993 to send instructions to desktop printers, not to store structured data for artificial intelligence.

A PDF file does not have a native concept of "paragraphs", "sentences", or "tables". It contains a collection of absolute visual instructions: draw character "T" at coordinates (x: 72, y: 150), draw a line from (100, 200) to (300, 200), and display an embedded JPEG image. When humans look at a rendered PDF, our visual cortex effortlessly reconstructs the columns, headers, and tables. But when a naive Python library extracts the text, it often produces an unreadable stream of jumbled characters.

If your RAG system ingests scrambled text, your embedding model creates corrupted vectors, and your LLM produces hallucinated answers. High-accuracy RAG begins with high-fidelity PDF parsing.

Production PDF parsing breaks down across three distinct layout patterns:

• 1. Multi-Column Reading Scrambling: In research papers and annual reports, text flows down Column 1, then continues at the top of Column 2. Naive extractors read left-to-right across the entire page, merging line 1 of Column 1 with line 1 of Column 2 into nonsensical composite sentences.
• 2. Running Header and Footer Pollution: Recurring page headers ("Confidential - Acme Corp 2025") and page numbers ("Page 43 of 90") appear every 300 words. If not stripped, these repeated snippets pollute your vector database and dilute semantic similarity scores.
• 3. Tabular Data Destruction: A table listing financial balances or technical specifications loses its row-and-column grid when extracted as plain text, turning structured numbers into an uninterpretable list of floating digits.

A reliable production pipeline processes PDF files through four sequential filters before any text touches an embedding model.

• Step 1: Digital vs Scanned Classification: Inspect the PDF to determine whether it contains a native digital text stream or is a scanned raster image. Digital PDFs are routed to fast text parsers (PyMuPDF); scanned pages are routed to an OCR pipeline.
• Step 2: Margin Bounding Box Cropping: Crop the top 8% and bottom 8% of each page bounding box. This automatically strips 99% of recurring headers, footers, and page numbers without complex regex patterns.
• Step 3: Column-Aware Text Sorting: Sort extracted text blocks vertically within detected column boundaries (`blocks.sort(key=lambda b: (b[0], b[1]))`), preserving the author’s intended reading flow.
• Step 4: Layout-Aware Table Isolation: Extract tables into structured Markdown tables (`| Col 1 | Col 2 |`) and treat them as distinct, indivisible chunks so table rows are never split across vector boundaries.

Many legacy documents — such as signed contracts, scanned invoices, and historical public filings — contain no digital text layer. Running standard text extractors returns an empty string.

For scanned documents, use a high-accuracy OCR engine paired with document layout analysis (like PaddleOCR or AWS Textract). The engine identifies text boxes, assigns confidence scores, and produces clean bounding-box coordinates.

For high-value, highly visual documents (such as investor decks or scientific patent applications), consider skipping text extraction entirely and deploying a visual retrieval model like ColPali, which embeds the rendered page image directly.

• Always check if a PDF has a selectable text layer before triggering expensive OCR models
• Crop header and footer margins to prevent vector index noise
• Convert extracted tables into structured Markdown before embedding
• Attach source page numbers and document titles as metadata to every chunk for clear user citations