Anyscale and MongoDB Collaborate to Enhance Multi-Modal Search

Anyscale,
a
leading
AI
application
platform,
has
announced
a
collaboration
with
MongoDB
to
improve
multi-modal
search
capabilities,
according
to
Anyscale.
This
partnership
aims
to
address
the
limitations
of
traditional
search
systems
and
provide
a
more
sophisticated
search
experience
for
enterprises
dealing
with
large
volumes
of
multi-modal
data.

Challenges
with
Legacy
Search
Systems

Enterprises
often
struggle
with
legacy
search
systems
that
are
not
equipped
to
handle
the
complexities
of
multi-modal
data,
which
includes
text,
images,
and
structured
data.
Traditional
systems
typically
rely
on
lexical
search
methods
that
match
text
tokens,
resulting
in
poor
recall
and
irrelevant
search
results.

For
instance,
an
e-commerce
platform
searching
for
a
“green
dress”
might
return
items
like
“Bio
Green
Apple
Shampoo”
due
to
the
limitations
of
lexical
search.
This
is
because
the
search
system
only
matches
text
tokens
and
does
not
understand
the
semantic
meaning
behind
the
query.

Innovative
Solution
Using
Anyscale
and
MongoDB

The
collaboration
between
Anyscale
and
MongoDB
aims
to
overcome
these
limitations
by
leveraging
advanced
AI
models
and
scalable
data
indexing
pipelines.
The
solution
involves:

• Using
  Anyscale
  to
  run
  multi-modal
  large
  language
  models
  (LLMs)
  to
  generate
  product
  descriptions
  from
  images
  and
  names.

• Generating
  embeddings
  for
  product
  names
  and
  descriptions,
  which
  are
  then
  indexed
  into
  MongoDB
  Atlas
  Vector
  Search.

• Creating
  a
  hybrid
  search
  backend
  that
  combines
  legacy
  text
  matching
  with
  advanced
  semantic
  search
  capabilities.

This
approach
enhances
the
search
relevance
and
user
experience
by
understanding
the
semantic
context
of
queries
and
returning
more
accurate
results.

Use
Case:
E-commerce
Platform

An
example
use
case
is
an
e-commerce
platform
with
a
large
catalog
of
products.
The
platform
aims
to
improve
its
search
capabilities
by
implementing
a
scalable
multi-modal
search
system
that
can
handle
both
text
and
image
data.
The
dataset
used
for
this
implementation
is
the
Myntra
dataset,
which
contains
images
and
metadata
of
products
for
Myntra,
an
Indian
fashion
e-commerce
company.

The
legacy
search
system
only
matched
text
tokens,
resulting
in
irrelevant
search
results.
By
using
Anyscale
and
MongoDB,
the
platform
can
now
return
more
relevant
results
by
understanding
the
semantic
meaning
of
queries
and
using
images
to
enrich
the
search
context.

System
Architecture

The
system
is
divided
into
two
main
stages:
an
offline
data
indexing
stage
and
an
online
search
stage.
The
data
indexing
stage
processes,
embeds,
and
upserts
text
and
images
into
MongoDB,
while
the
search
stage
handles
search
requests
in
real-time.

Data
Indexing
Stage

This
stage
involves:

• Metadata
  enrichment
  using
  multi-modal
  LLMs
  to
  generate
  product
  descriptions
  and
  metadata
  fields.

• Embedding
  generation
  for
  product
  names
  and
  descriptions.

• Data
  ingestion
  into
  MongoDB
  Atlas
  Vector
  Search.

Search
Stage

The
search
stage
combines
legacy
text
matching
with
advanced
semantic
search.
It
involves:

1. Sending
   a
   search
   request
   from
   the
   frontend.

2. Processing
   the
   request
   at
   the
   ingress
   deployment.

3. Generating
   embeddings
   for
   the
   query
   text.

4. Performing
   a
   vector
   search
   on
   MongoDB.

5. Returning
   the
   search
   results
   to
   the
   frontend.

Conclusion

The
collaboration
between
Anyscale
and
MongoDB
represents
a
significant
advancement
in
multi-modal
search
technology.
By
integrating
advanced
AI
models
and
scalable
data
indexing
pipelines,
enterprises
can
now
offer
a
more
relevant
and
efficient
search
experience.
This
solution
is
particularly
beneficial
for
e-commerce
platforms
looking
to
improve
their
search
capabilities
and
user
experience.

For
more
information,
visit
the

Anyscale
blog.

Image
source:
Shutterstock