NVIDIA Unveils First On-Device Small Language Model for Enhanced Game Character Roleplay

At
Gamescom
2024,
NVIDIA
announced
the
deployment
of
its
first
on-device
small
language
model
(SLM)
aimed
at
enhancing
the
conversational
abilities
of
game
characters,
according
to
the

NVIDIA
Technical
Blog.
This
innovation
is
set
to
debut
in
Amazing
Seasun
Games’ ‘Mecha
BREAK’,
promising
a
more
dynamic
and
immersive
gameplay
experience
on
NVIDIA
GeForce
RTX
AI
PCs.

NVIDIA
ACE
Introduces
SLM
for
Roleplaying

NVIDIA
ACE,
a
suite
of
digital
human
technologies,
now
includes
the
first
NVIDIA
SLM,
Nemotron-4
4B
Instruct.
This
model
is
available
as
an
NVIDIA
NIM
for
both
cloud
and
on-device
deployment,
offering
game
developers
a
streamlined
way
to
integrate
advanced
language
models
into
their
games.
NVIDIA
NIM
provides
a
set
of
microservices
for
faster
deployment
of
these
models
on
any
cloud
or
data
center.

Nemotron-4
4B
Instruct
is
optimized
through
distillation,
pruning,
and
INT4
quantization,
minimizing
VRAM
usage
to
around
2
GB
and
significantly
reducing
the
time-to-first-token
compared
to
larger
language
models
(LLMs).
The
model,
distilled
from
the
Nemotron-4
15B
LLM,
is
fine-tuned
with
instructional
prompts
for
better
performance
in
specific
tasks,
enhancing
role-play,
retrieval
augmented
generation
(RAG),
and
function-calling
capabilities.

This
technology
allows
game
characters
to
more
intuitively
comprehend
player
instructions,
respond
accurately,
and
perform
relevant
actions.
Developers
can
integrate
Nemotron-4
4B
Instruct
NIM
into
their
digital
human
technology
pipeline
via
the
cloud
or
leverage
over
100
million
NVIDIA
GeForce
RTX-powered
PCs
and
laptops
for
on-device
inference.
Additionally,
the
Unreal
Engine
5
ACE
sample
application
has
been
updated
to
support
on-device
plugin
integration.

First
Game
Premiere
with
NVIDIA
ACE
NIM

‘Mecha
BREAK’,
developed
by
Amazing
Seasun
Games,
a
subsidiary
of
Kingsoft
Corporation,
is
the
first
game
to
showcase
the
NVIDIA
Nemotron-4
4B
Instruct
NIM
running
on-device.
The
game
leverages
NVIDIA’s
Audio2Face-3D
NIM
and
Whisper,
OpenAI’s
speech
recognition
model,
for
facial
animation
and
speech
recognition,
while
Elevenlabs
provides
the
characters’
voices
through
cloud
services.

In
this
demo,
NVIDIA
ACE
and
digital
human
technologies
enable
players
to
interact
with
a
mechanic
character,
who
assists
in
mech
selection,
customization,
and
preparation
for
battles,
enhancing
the
overall
gaming
experience.

Game
Developers
Explore
Latest
Digital
Human
Technologies

NVIDIA
ACE
and
digital
human
technologies
are
expanding
their
influence
in
the
gaming
industry.
Perfect
World
Games
is
advancing
its
NVIDIA
ACE
and
digital
human
technology
demo, ‘Legends’,
with
new
AI-powered
vision
capabilities.
In
this
demo,
the
character
Yun
Ni
can
see
gamers
and
identify
people
and
objects
in
the
real
world
using
the
computer’s
camera,
powered
by
ChatGPT-4o,
adding
an
augmented
reality
layer
to
the
gameplay.

These
advancements
promise
a
new
level
of
immersion
and
accessibility
for
PC
games.
For
more
information,
visit

NVIDIA
ACE
and
download
the
NIM
to
start
building
game
characters
powered
by
generative
AI.

Image
source:
Shutterstock