Snap, Search, Sip: CLIP-Powered Coffee Discovery

Image-Based Coffee Search for a Social Brewing Platform

Executive Summary

The product and AI engineering team supported a European coffee community in turning product discovery into a snap-and-search experience. Instead of typing brand and roast names on mobile, users upload a coffee bag image and immediately receive exact matches or near-duplicates—plus visually and semantically similar roasts—to inspire exploration. This CLIP-powered visual search was embedded into a broader platform for shots, recipes, and reviews, with role-based portals for admins and roasters to curate catalogs and manage engagement.

Problem

Typing on mobile is slow and error-prone, especially across thousands of roasts, origins, and label designs. A search experience that relies on text alone misses the visual cues that define specialty coffee packaging and brand identity. The platform required a discovery layer that recognizes a bag from a single photo, returns confident matches in real time, and generalizes to new products without heavyweight retraining—while integrating cleanly into iOS/Android clients and the existing backend.

Solution

The AI team implemented a contrastive-embedding pipeline using CLIP/Jina-CLIP. Product images were cleaned, deduplicated, and standardized; embeddings were generated and indexed for approximate nearest-neighbor retrieval. The platform exposes a stateless inference API that accepts a user photo, performs embedding, queries the vector index, and returns top-k candidates with similarity scores and metadata. The integration delivers two complementary experiences: (1) “Exact or near-exact match” for quick identification, and (2) “Similar coffees” for discovery by visual/semantic proximity (origin, roast level, processing hints present in label art or descriptors). Mobile clients receive compact payloads for fast rendering, and admin/roaster portals include tooling to review matches, suppress false positives, and append new SKUs without model retraining. Test coverage with Pytest validates edge cases (glare, crop, blur), and CI/CD deploys to Hugging Face Inference and Jina endpoints for elastic scaling.

Outcome

A fully functional, low-latency image search went live in eight weeks, reducing friction in product lookup and increasing session depth through visually driven discovery. Catalog expansion no longer demands retraining—new coffees join via embeddings-only workflows—keeping operating costs predictable. The result is a consumer experience that feels immediate and a roaster ecosystem that stays current as offerings evolve.

What You Can Expect Working with Dreamloop Studio

Dreamloop Studio’s product and AI teams specialize in turning modern embedding models into delightful, shippable features. Expect pragmatic pipelines, fast mobile integrations, and production-grade MLOps that scale with your catalog. The outcome is a visual discovery layer that users trust, partners can manage, and finance teams can forecast.