Google Cloud Run Integrates NVIDIA L4 GPUs for Enhanced AI Inference Deployments

Google
Cloud
Run
has
announced
the
integration
of
NVIDIA
L4
Tensor
Core
GPUs,
NVIDIA
NIM
microservices,
and
capabilities
for
serverless
AI
inference
deployments,
according
to
the

NVIDIA
Technical
Blog.
This
collaboration
aims
to
address
the
challenges
enterprises
face
when
deploying
AI-enabled
applications,
including
performance
optimization,
scalability,
and
infrastructure
complexity.

Enhancing
AI
Inference
Deployments

Google
Cloud’s
fully
managed
serverless
container
runtime,
Cloud
Run,
now
supports
NVIDIA
L4
Tensor
Core
GPUs
in
preview.
This
allows
enterprises
to
run
real-time
AI
applications
on
demand
without
the
hassle
of
managing
infrastructure.
The
integration
of
NVIDIA
NIM
microservices
further
simplifies
the
optimization
and
deployment
of
AI
models,
maximizing
application
performance
and
reducing
complexity.

Real-Time
AI-Enabled
Applications

Cloud
Run
abstracts
infrastructure
management
by
dynamically
allocating
resources
based
on
incoming
traffic,
ensuring
efficient
scaling
and
resource
utilization.
The
support
for
NVIDIA
L4
GPUs
represents
a
significant
upgrade
from
previous
CPU-only
offerings,
providing
up
to
120x
higher
AI
video
performance
over
CPU
solutions
and
2.7x
more
generative
AI
inference
performance
over
the
previous
generation.

Notably,
companies
like
Let’s
Enhance,
Wombo,
Writer,
Descript,
and
AppLovin
are
leveraging
NVIDIA
L4
GPUs
to
power
their
generative
AI
applications,
delivering
enhanced
user
experiences.

Performance-Optimized
Serverless
AI
Inference

Optimizing
AI
model
performance
is
crucial
for
resource
efficiency
and
cost
management.
NVIDIA
NIM
offers
a
set
of
optimized
cloud-native
microservices
that
simplify
and
accelerate
AI
model
deployment.
These
pre-optimized,
containerized
models
integrate
seamlessly
into
applications,
reducing
development
time
and
maximizing
resource
efficiency.

NVIDIA
NIM
on
Cloud
Run
allows
for
the
deployment
of
high-performance
AI
applications
using
optimized
inference
engines
that
unlock
the
full
potential
of
NVIDIA
L4
GPUs,
providing
superior
throughput
and
latency
without
requiring
specialized
expertise
in
inference
performance
optimization.

Deploying
Llama3-8B-Instruct
NIM
Microservice

Deploying
models
like
Llama3-8B-Instruct
with
Cloud
Run
on
NVIDIA
L4
GPUs
is
straightforward.
Users
need
to
install
the
Google
Cloud
SDK
and
follow
a
series
of
steps
to
clone
the
repository,
set
environment
variables,
edit
the
Dockerfile,
build
the
container,
and
deploy
it
using
provided
scripts.

Getting
Started

The
integration
of
the
NVIDIA
AI
platform,
including
NVIDIA
NIM
and
NVIDIA
L4
GPUs,
with
Google
Cloud
Run
addresses
key
challenges
in
AI
application
deployment.
This
synergy
accelerates
deployment,
boosts
performance,
and
ensures
operational
efficiency
and
cost-effectiveness.

Developers
can
prototype
with
NVIDIA
NIM
microservices
through
the
NVIDIA
API
catalog,
then
download
NIM
containers
for
further
development
on
Google
Cloud
Run.
For
enterprise-grade
security
and
support,
a
90-day
NVIDIA
AI
Enterprise
license
is
available.

Currently,
Cloud
Run
with
NVIDIA
L4
GPU
support
is
in
preview
in
the
us-central1
Google
Cloud
region.
More
information
and
demos
are
available
at
the
launch
event
livestream
and
sign-up
page.

Image
source:
Shutterstock