NVIDIA Maxine Unveils Next-Gen Digital Humans and Telepresence Innovations at SIGGRAPH 2024

Rebeca
Moen

Jul
30,
2024
03:40

NVIDIA
Maxine
introduces
groundbreaking
advancements
in
telepresence
and
digital
human
technology,
featuring
Maxine
3D
and
Video
Relighting
at
SIGGRAPH
2024.

At
SIGGRAPH
2024,
NVIDIA
showcased
the
latest
advancements
in
its
Maxine
AI
developer
platform,
available
through
NVIDIA
AI
Enterprise.
This
platform
is
designed
to
enhance
audio
and
video
quality
and
enable
augmented
reality
effects.

New
Features
and
Enhancements

NVIDIA
announced
the
upcoming
availability
of
Maxine
3D
and
Maxine
Video
Relighting
for
early
access
developers,
alongside
the
production
launch
of
the
Maxine
Eye
Contact
microservice.
These
innovations
aim
to
bring
true-to-life
digital
humans
and
immersive
telepresence
experiences
within
reach
of
a
wide
range
of
applications.

Maxine
3D,
in
conjunction
with
NVIDIA
ACE,
a
suite
of
generative
AI
technologies,
enables
real-time,
photoreal
3D
avatars
using
standard
video-conferencing
devices.
The
Eye
Contact
and
Audio2Face-2D
(also
known
as
Speech
Live
Portrait)
features
are
now
accessible
through
the
NVIDIA
API
Catalog,
offering
enhanced
discoverability
and
trial
options.

Groundbreaking
Technologies

Maxine
3D
stands
out
for
its
ability
to
convert
2D
video
portrait
inputs
into
immersive
3D
avatars
in
real
time.
This
technology
integrates
with
NVIDIA
RTX
rendering
to
provide
lifelike
visuals,
transforming
standard
2D
video
inputs
into
dynamic
3D
avatars.
Shawn
Frayne,
co-founder
and
CEO
of
Looking
Glass,
highlighted
Maxine’s
potential
to
realize
virtual
teleportation
between
physical
spaces.

Looking
Glass
has
been
collaborating
with
NVIDIA
Research
to
create
an
innovative
video
conferencing
showcase
using
holographic
3D
displays.
This
partnership
utilizes
NVIDIA
RTX
6000
Ada
GPUs
and
Maxine
3D
to
enable
multiple
viewers
to
experience
authentic
3D
content
simultaneously
without
the
need
for
headsets
or
eye
tracking.

Enhanced
Discoverability
and
Accessibility

NVIDIA
has
introduced
Maxine
features
to
its
API
Catalog,
allowing
developers
to
explore
and
trial
cutting-edge
capabilities
easily.
These
features
are
also
available
as
NVIDIA
NIM
microservices,
offering
a
highly
optimized
solution
for
AI
deployment
with
prebuilt
containers
and
industry-standard
APIs.

As
part
of
the
NVIDIA
AI
Enterprise
software
platform,
these
microservices
come
with
rigorous
validation,
security
updates,
and
enterprise
support,
making
them
ideal
for
businesses
seeking
robust
solutions.

Advanced
Video
and
Audio
Enhancements

Several
new
and
enhanced
features
are
being
introduced
to
improve
the
user
experience:

• Video
  Relighting
• Studio
  Voice
• Background
  Noise
  Reduction
  2.0
• Maxine
  hosted
  APIs

Video
Relighting

The
Maxine
Video
Relighting
microservice,
currently
in
Early
Access,
uses
AI
to
match
foreground
illumination
with
various
backgrounds
and
environments
in
real
time.
This
ensures
subjects
always
look
their
best,
regardless
of
their
physical
environment.

Studio
Voice

The
latest
iteration
of
Studio
Voice
offers
significant
improvements
in
quality
and
performance,
making
it
viable
for
real-time
communications
and
bringing
studio-quality
audio
to
everyday
video
conferencing
setups.

Background
Noise
Reduction
2.0

This
feature
sets
a
new
standard
in
audio
clarity,
effectively
eliminating
background
noise
while
preserving
the
natural
quality
of
speech.
It
is
particularly
useful
when
combined
with
automatic
speech
recognition
(ASR)
technology
to
reduce
transcription
errors.

Empowering
Developers
and
Industries

NVIDIA
Maxine
is
a
comprehensive
platform
that
enables
the
creation
of
next-generation
applications
for
telepresence
and
digital
human
creation.
It
provides
tools
that
empower
industries
ranging
from
entertainment
and
gaming
to
healthcare
and
education.

As
virtual
influencers,
AI
assistants,
and
digital
avatars
become
more
prevalent,
Maxine’s
technologies
offer
the
foundation
for
creating
believable
and
engaging
digital
personas.

Looking
Ahead

SIGGRAPH
2024
demonstrated
that
NVIDIA
Maxine
is
set
to
play
a
pivotal
role
in
the
future
of
digital
communication
and
telepresence.
With
its
advanced
AI
capabilities
and
focus
on
developer
accessibility,
the
Maxine
developer
platform
is
poised
to
enable
new
possibilities
for
interaction
in
digital
spaces.

The
combination
of
Maxine
3D,
advanced
audio-visual
enhancements,
and
easy-to-integrate
APIs
positions
NVIDIA
partners
at
the
forefront
of
the
digital
human
revolution.
As
the
market
for
these
technologies
grows,
NVIDIA
innovations
are
set
to
enable
the
next
wave
of
immersive,
lifelike
digital
experiences
across
industries.

For
more
information,
visit
the

official
NVIDIA
blog.

Image
source:
Shutterstock