What Happened
Perplexity released two pplx-embed-v2-late models that search PDF pages as images using late interaction. They can represent tables, charts, and page layout without an OCR step during indexing [1]. This addresses a common weakness of text-only retrieval: a financial figure is easy to misinterpret when its row, column, unit, or label is missing [1].
Why It Matters to Businesses
For reports, statements, and research PDFs, finding the right document is not enough. An answer may still be wrong if retrieval loses the visual relationship between a value and its heading. Page-image search offers another way to find relevant evidence, but retrieval alone does not prove that an extracted number or its interpretation is correct.
Kimbodo Engineering Perspective
Use visual retrieval selectively, not as a default replacement for text search. Text and keyword indexes remain useful for exact terms, identifiers, and filters. Page-image retrieval is most valuable where layout carries meaning. Late-interaction retrieval can also increase index and query costs, so its benefit should be measured against a representative document set before broad rollout.
LlamaIndex, LangChain, and Haystack can orchestrate ingestion and retrieval. Weaviate, Pinecone, Qdrant, Milvus, Elasticsearch, and Vespa are infrastructure options to evaluate against the chosen retrieval pattern. The deciding criteria are the required vector representation, filtering, scale, latency, operational burden, and access controls—not framework preference.
How We Would Implement It
- Store each original PDF and render pages with stable document, page, and version identifiers.
- Build a baseline hybrid index combining extracted text, keyword search, metadata filters, and text embeddings. Add a page-image retrieval index for layout-heavy documents [1].
- Retrieve candidates through both paths, rerank them, and pass the selected page images or verified excerpts to the answering model with page-level citations.
- Evaluate on questions that require reading table headings, chart labels, and units. Track retrieval recall, answer correctness, citation accuracy, latency, and cost before changing the production routing policy.
Risks, Costs and Security
Image indexing and late-interaction scoring can raise storage and compute costs; rendering and document updates add operational work. OCR-free indexing does not eliminate the need to verify values used in an answer. Apply document-level permissions before retrieval, enforce tenant isolation across indexes and caches, and retain enough provenance to trace every cited answer to the authorized page.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our RAG Development Services practice, or Estimate My RAG System.
