Perplexity Open-Sources New Retrieval Model: 0.6B Small Model Can Directly Query Indexes Built by 9B

Beating AI News Flash: Perplexity has open-sourced two multimodal embedding models, pplx-embed-v2-late, at 0.6B and 9B respectively. The vectors of the two models are mutually compatible, allowing developers to build a knowledge base with the 9B large model and then use the 0.6B small model for everyday search, eliminating the need to run the large model for every query.

The new models also support retrieval of text, images, and PDF pages. Conventional embedding models typically compress a piece of content into a single vector, which easily loses details. The new models retain a 128-dimensional vector for each token, allowing search terms to separately match relevant content within documents. When processing PDFs, PPTs, and scanned documents, page images can be directly converted into vectors without first using OCR to extract text, while also preserving charts, tables, and layout information.

The two models were trained from the same 18B teacher model and share a single vector space. In official ViDoRe v3 image retrieval tests, building the database and querying with the 0.6B model scored 62.3%; switching to 9B for database building and 0.6B for querying raised the score to 63.5%. Using the 9B model entirely scored 65.2%.

The weights of both models have been released on Hugging Face under the MIT license.