What Happened
DMIS Lab’s UltraBERT-Embed-150M is a 151M-parameter English embedding model that accepts inputs up to 32K tokens and produces one 768-dimensional vector per input. That makes a single-vector representation possible for many documents that would otherwise be split into chunks. A Qdrant test compares document-level vectors, 512-token chunks, a combination of both, and compressed versions of each. The research note describes the comparison, not a result showing that chunking is obsolete [1].
Why It Matters to Businesses
Fewer vectors could simplify indexing and reduce storage and query overhead. But a document-level match may not identify the passage needed to answer a specific question. For RAG applications, the business measure is not how few vectors an index holds; it is whether users receive accurate answers with traceable evidence at an acceptable latency and cost.
Kimbodo Engineering Perspective
We would treat long-input embeddings as another retrieval option, not a default replacement for chunks. Document-level vectors are useful for broad discovery; passage-level vectors are better suited to finding precise evidence. A hybrid design can retrieve candidate documents first, then search or rerank passages within them. The right choice depends on document length, question specificity, update frequency and the quality of citations required.
LlamaIndex, LangChain and Haystack can help orchestrate ingestion and retrieval, while Qdrant, Weaviate, Pinecone, Milvus, Elasticsearch and Vespa are infrastructure options to assess against existing operations and search requirements. The embedding model, retrieval strategy and index should be evaluated separately rather than selected as one package.
How We Would Implement It
- Build a representative evaluation set: Include broad document-finding questions, precise fact-finding questions and queries that require evidence from multiple passages. Record the expected documents and supporting passages.
- Index comparable variants: Test one vector per document, 512-token chunks, and a combined document-plus-chunk index, mirroring the comparison described in [1]. Add keyword retrieval where exact terms, identifiers or names matter.
- Separate discovery from evidence: Retrieve candidate documents, identify supporting passages, and pass only those passages to the answer generator. Store document IDs, passage offsets and version metadata so answers can cite the material actually used.
- Measure the full workflow: Compare evidence recall, answer accuracy, citation accuracy, ingestion time, index size, query latency and operating cost. Test compressed indexes only if they preserve the retrieval quality the application needs.
Risks, Costs and Security
A single vector compresses a long document into one representation; important details may be harder to retrieve even when the document fits within the model’s input limit. Chunk-heavy indexes create more records to store, update and secure. Combined indexes can improve flexibility but add ingestion and operational complexity. Results from an English embedding model should not be assumed to hold for multilingual collections [1].
Apply access controls before retrieval results reach the model, including when document and passage indexes are queried separately. Track source versions, remove stale vectors when content changes, and treat retrieved text as untrusted input that may contain prompt-injection instructions. These controls matter more than a small reduction in vector count when the application handles sensitive business data.
Where Kimbodo Comes In
Kimbodo builds and operates this in production for businesses — see our RAG Development Services practice, or Estimate My RAG System.
