Executive summary
Short: A 2026 benchmark on a 5,000-item apparel & footwear catalog found that a simple average of image and text embeddings (L2-normalize each, mean, then L2-normalize) produced substantially better product search results than image-only or text-only retrieval—up to ~1.5× improvement on top-of-list metrics (Recall@1/5/10, MRR, nDCG@10). The study also showed multimodal averaging improved cross-language retrieval and that AI-generated descriptions underperformed good human text. Results varied by model: a CLIP‑style dual encoder outperformed a newer omni model on this catalog. Practical trade-offs include double embedding time for averaging, a need to reindex on model change, and licensing/deployment differences across models and endpoints [1].
Chronological timeline of key developments
- 2026-07-16 — Benchmark of product-search embeddings on 5,000 Kaggle apparel/footwear items:
- Designed deterministic, reproducible queries (including German variants) and compared image-only, text-only, and averaged image+text embeddings.
- Found averaged (L2(img)+L2(txt))/2 then L2 normalized gave up to ~1.5× better top-of-list retrieval than image-only; also improved cross-language retrieval.
- AI-generated descriptions performed worse than human-written text on a 1,000-item subset.
- CLIP-style jina-clip-v2 (~865M params) outperformed jina-embeddings-v5-omni-small (~1.74B params) on this catalog; omni model had weaker multimodal combination behavior and API limits on per-request modality embedding [1].
Trends
- Multimodal fusion (practical heuristics): Simple arithmetic fusion (average of normalized image + text vectors) is an effective baseline for product search and cross-language retrieval on catalog data [1].
- Model performance is not strictly monotonic: Newer or larger models do not always outperform older, purpose-built dual encoders in downstream retrieval for a given domain [1].
- Human text remains valuable: High-quality human descriptions can outperform AI-generated text for retrieval tasks [1].
- Operational trade-offs matter: Multimodal averaging doubles embedding calls (increased offline cost) but does not increase retrieval-time cost; reindexing is required when switching models [1].
- Reproducibility and deterministic query sets: Rule-based, seedable query generation with native-speaker checks enables stable benchmarking and cross-language checks [1].
Risks
- Overreliance on heuristics: Averaging is a simple heuristic, not a learned joint embedding; it may be outperformed by true multimodal models in some domains or future models [1].
- Domain sensitivity: Performance gains can flip by domain, photo style, or product category—benchmarks on other catalogs may not generalize [1].
- Operational cost and latency: Doubling embedding calls increases compute cost and pipeline complexity (though retrieval latency unaffected) [1].
- Degraded retrieval from AI-generated text: Replacing human product copy with model-generated descriptions can reduce retrieval quality [1].
- Model/version and licensing constraints: Different models and endpoints impose licensing, deployment, and per-request modality limits; switching models triggers full reindexing [1].
Opportunities
- Adopt multimodal averaging as a strong baseline: Implement L2-normalize each modality, mean, then L2-normalize to quickly improve retrieval without changing your retrieval stack [1].
- Improve cross-language support: Multimodal averaging can boost cross-language recall with modest effort; include multilingual text or translation checks in evaluation [1].
- Benchmark rigorously and reproducibly: Use deterministic query families, seedable generation, and small validation subsets (including human vs AI text) to guide model selection and QA [1].
- Cost engineering: Move multimodal embedding to offline pipelines, batch embeddings, and cache vectors to absorb the extra compute cost while keeping retrieval fast [1].
- Model selection by task and domain: Test both dual-encoder CLIP-style and newer omni/shared-backbone models on your catalog rather than assuming larger models are better [1].
Recommended actions
- Run a small pilot (recommended minimum): On a representative subset (≈1–5k items), produce image-only, text-only, and averaged image+text vectors; evaluate Recall@1/5/10, MRR, nDCG@10 and cross-language queries to verify gains on your data [1].
- Implement averaging exactly as tested: L2-normalize image vector, L2-normalize text vector, compute mean, then L2-normalize the result; index the averaged vectors alongside single-modality indexes for ablation [1].
- Preserve human descriptions: Maintain high-quality human copy in the catalog and include a test split comparing human vs AI-generated descriptions before replacing anything [1].
- Plan for embedding cost and reindexing: Treat averaging as two embedding calls per item (plan offline batch jobs, incremental reindexing strategies, and budget for reindex runs when models change) [1].
- Track model metadata and licensing: Record model name, version, date, API/endpoint, and license; confirm deployment and usage constraints before production rollout [1].
- Automate reproducible evaluation: Build deterministic query generators (including multilingual variants), store gold sets, and include them in CI/benchmark pipelines to detect regressions after model or pipeline changes [1].
- Compare architectures for your domain: Evaluate both CLIP-style dual encoders and omni/shared-backbone multimodal models; do not assume size predicts retrieval performance—measure on your catalog [1].
