NVIDIA Unveils NeMo Retriever Microservices to Enhance AI Accuracy and Throughput

NVIDIA
has
announced
the
launch
of
its
new
NeMo
Retriever
NIM
(NeMo
Inference
Microservices)
designed
to
significantly
enhance
the
accuracy
and
throughput
of
Large
Language
Models
(LLMs)
in
AI
applications.
These
microservices
aim
to
help
developers
access
and
utilize
proprietary
data
more
efficiently,
thereby
generating
more
accurate
and
relevant
responses
for
AI-driven
tasks,
according
to
the

NVIDIA
Blog.

Boosting
AI
Accuracy
with
NeMo
Retriever

The
NeMo
Retriever
NIM
microservices
are
production-ready
and
designed
for
retrieval-augmented
generation
(RAG).
This
new
suite
of
tools
allows
enterprises
to
scale
AI
workflows
with
minimal
intervention,
ensuring
high
accuracy
in
various
applications.
The
microservices
integrate
seamlessly
with
platforms
such
as
Cohesity,
DataStax,
NetApp,
and
Snowflake.

These
microservices
are
particularly
beneficial
for
developers
working
on
AI
agents,
customer
service
chatbots,
security
vulnerability
analysis,
and
extracting
insights
from
complex
supply
chain
data.
By
enabling
high-performance,
enterprise-grade
inferencing,
NeMo
Retriever
NIM
microservices
can
supercharge
AI
applications
with
enhanced
data
accuracy
and
throughput.

Embedding
and
Reranking
Models

NeMo
Retriever
NIM
microservices
consist
of
two
main
model
types:
embedding
and
reranking.
Embedding
models
transform
diverse
data
into
numerical
vectors,
capturing
their
meaning
and
nuances,
while
reranking
models
score
data
based
on
its
relevance
to
a
given
query.
By
combining
these
two
models,
developers
can
ensure
the
most
accurate
and
relevant
results
for
their
AI
applications.

The
embedding
models,
such
as
NV-EmbedQA-E5-v5
and
NV-EmbedQA-Mistral7B-v2,
are
optimized
for
text
question-answering
retrieval
and
multilingual
embedding,
respectively.
The
reranking
models,
including
NV-RerankQA-Mistral4B-v3,
provide
high-accuracy
text
reranking
capabilities.
These
models
are
now
generally
available
and
accessible
through
the
NVIDIA
API
catalog.

Top
Use
Cases

NeMo
Retriever
NIM
microservices
offer
a
wide
range
of
applications,
from
building
intelligent
chatbots
and
analyzing
security
vulnerabilities
to
extracting
insights
from
supply
chain
information
and
enhancing
retail
shopping
advisors.
These
microservices
are
also
being
integrated
by
various
partners
to
boost
the
accuracy
and
throughput
of
their
AI
models.

For
instance,
DataStax
has
incorporated
NeMo
Retriever
embedding
NIM
microservices
into
its
Astra
DB
and
Hyper-Converged
platforms,
while
Cohesity
is
integrating
these
microservices
with
its
AI
product,
Cohesity
Gaia.
NetApp
is
collaborating
with
NVIDIA
to
connect
NeMo
Retriever
microservices
to
its
intelligent
data
infrastructure,
enabling
seamless
access
to
business
insights
without
compromising
data
security.

Integration
with
Other
NIM
Microservices

NeMo
Retriever
NIM
microservices
can
be
used
alongside
other
NVIDIA
microservices,
such
as
NVIDIA
Riva
NIM,
which
enhances
speech
AI
applications.
Upcoming
models
like
FastPitch
and
HiFi-GAN
for
text-to-speech
applications,
and
Megatron
for
multilingual
neural
machine
translation,
will
soon
be
available
as
Riva
NIM
microservices.

These
microservices
can
be
deployed
in
various
environments,
including
cloud
instances
from
major
providers
like
AWS,
Google
Cloud,
Microsoft
Azure,
and
Oracle
Cloud
Infrastructure.
They
can
also
run
on
NVIDIA-Certified
Systems
from
server
manufacturing
partners
like
Cisco,
Dell
Technologies,
Hewlett
Packard
Enterprise,
Lenovo,
and
Supermicro.

Members
of
the
NVIDIA
Developer
Program
will
soon
have
free
access
to
NIM
for
research,
development,
and
testing
on
their
preferred
infrastructure.
Enterprises
can
deploy
these
microservices
in
production
through
the
NVIDIA
AI
Enterprise
software
platform.

Image
source:
Shutterstock