NVIDIA Holoscan and RTI Connext Pave the Way for AI-Enabled Medical Devices

The
demand
for
real-time
insights
and
autonomous
decision-making
is
growing
across
industries,
and
healthcare
and
medical
devices
are
no
exception.
Leveraging
real-time
edge
AI,
the
next
generation
of
healthcare
promises
to
deliver
more
precise
treatments,
improve
patient
outcomes,
and
increase
operational
efficiencies.

Operating
rooms
of
the
future
will
increasingly
incorporate
AI-enabled
and
interconnected
devices,
providing
real-time
access
to
holistic
patient
data,
operative
insights,
decisions,
and
actions.
In
such
a
future,
software
as
a
medical
device
(SaMD)
must
handle
large-scale
data
with
stringent
performance
and
latency
constraints
while
deployed
in
distributed
healthcare
systems.
This
requires
interoperability
to
ensure
efficient,
reliable,
and
secure
data
connectivity
between
various
sensors,
displays,
controls,
and
applications
without
compromising
performance
or
latency.

This
post
demonstrates
how
to
integrate

NVIDIA
Holoscan
and

RTI
Connext
to
create
AI-powered
medical
device
applications
with
high
interoperability,
low
latency,
and
distributed
connectivity.
This
integration
achieves
these
benefits
with
minimal
overhead
and
reduced
implementation
effort
and
complexity.

NVIDIA
Holoscan
for
Real-Time
AI
Sensor
Processing

NVIDIA
Holoscan
provides
developers
with
a
production-ready
framework
for
building
an
end-to-end
real-time
AI
sensor
processing
pipeline,
from
sensor
data
ingress
to
accelerated
computing
and
AI
inferencing,
real-time
visualization,
actuation,
and
data
stream
egress.
This
comprehensive
solution
effectively
addresses
the
multitude
of
edge
AI
development
challenges,
ensuring
optimal
application
performance
while
abstracting
away
development
complexities,
reducing
time
to
market,
and
offering
the
convenience
of
coding
in
Python
and
C++.

RTI
Connext
for
Real-Time
Data-Centric
Connectivity

RTI
Connext,
based
on
the
Data
Distribution
Service
(DDS)
standard,
streamlines
connectivity
across
complex
and
scalable
systems
with
a
distributed
and
real-time
software
communication
framework.
With
Connext,
Holoscan
applications
can
integrate
with
distributed
data
sources
and
applications
with
little
overhead
while
minimizing
the
implementation
effort
and
complexity
of
achieving
the
performance,
reliability,
and
security
required
by
healthcare
systems.

Connext
provides
real-time
information
exchange
between
complex
system
components,
enabling
stringent
reliability,
cybersecurity,
and
performance
requirements.
Medical
systems
built
on
Connext
are
resilient,
self-forming,
and
self-healing
with
no
single
point
of
failure.
The
wide
range
of
quality
of
service
tuning
options
helps
meet
the
need
for
real-time
video
and
correlated
data
in
distributed,
intelligent
surgical
systems.
Built-in
security
based
on
the
DDS
Security
standard
provides
the
foundation
for
authentication
and
encryption,
as
well
as
security
logging
and
granular
access
control,
keeping
critical
systems
safe
from
security
breaches
and
meeting
cybersecurity
requirements
enforced
by
regulatory
agencies
such
as
the
FDA.

Integrating
NVIDIA
Holoscan
and
RTI
Connext

Today’s
healthcare
system
is
built
on
numerous
installed
legacy
systems
that
weren’t
originally
designed
with
AI
capabilities
in
mind
and
where
NVIDIA
Holoscan
is
not
currently
natively
supported.
The
integration
of
Connext
with
Holoscan
enables
developers
to
transform
existing
legacy
installations
to
AI-enabled
and
software-defined
devices
through
integration
of
Holoscan
as
a
sidecar
(companion
compute
module)
to
those
devices
where
Holoscan
is
not
natively
supported.

For
example,
many
existing
medical
devices
are
currently
Windows-based,
particularly
in
the
domain
of
medical
imaging,
where
Holoscan
is
not
natively
supported.
Holoscan
as
a
sidecar
could
bring
advanced
AI
capabilities
into
robotic
surgery
systems
running
real-time
operating
systems
(RTOS)
on
non-NVIDIA
systems.
Additionally,
low-end
sensory
medical
devices
such
as
patient
monitoring
could
be
augmented
with
powerful
AI
algorithms
while
the
legacy
system’s
hardware
or
software
would
otherwise
limit
the
addition
of
such
new
capabilities.

Holoscan
DDS
interoperability
through
RTI
Connext
DDS
offers
a
solution
to
these
scenarios,
providing
a
scalable,
AI-enabled
Holoscan
sidecar
that
seamlessly
communicates
with
legacy
systems
in
real
time.
Holoscan
offers
exceptional
infrastructure
for
GPU-accelerated
SaMD,
enabling
the
innovation
and
deployment
of
AI-powered
workflows
within
next-generation
healthcare
systems,
which
generally
need
to
operate
on
large-scale
data
with
very
strict
latency
restrictions.

Using
RTI
Connext,
Holoscan
applications
can
integrate
with
distributed
healthcare
systems
with
little
overhead
while
minimizing
the
implementation
effort
and
complexity
of
achieving
the
performance,
reliability,
and
security
required
by
such
systems.
In
cases
where
Connext
is
already
being
used,
introducing
new
Holoscan-powered
AI
workflows
may
be
possible
without
modifications
to
the
existing
system.

Example
Holoscan
Application
with
RTI
Connext
Integration

This
section
provides
an
example
use
case:
a
Holoscan
application
running
on
a
dedicated
system
that
acts
as
a
sidecar.
The
application
reads
frames
from
a
DDS
databus
using
RTI
Connext,
processes
the
frame
data
within
a
Holoscan
workflow,
and
then
publishes
the
results
back
to
the
databus
with
Connext
so
the
processed
frame
data
can
be
read
by
another
device
for
display.

This
example
enhances
a
common
scenario
in
healthcare
systems
in
which
multiple
sensors
capture
data
that
is
then
aggregated
for
display
on
a
separate
monitoring
system.
Adding
AI-powered
Holoscan
workflows
into
the
middle
of
this
data
flow
can
often
be
done
with
little
modification
to
the
existing
components.
Connext
helps
to
bridge
these
gaps.

The
core
components
of
this
example
are
the

Holoscan
DDS
video
streaming
operators
available
through

nvidia-holoscan/holohub
on
GitHub.
These
operators
enable
Holoscan
applications
to
read
and
write
video
frames
from
a
DDS
databus
in
real
time.
With
these
operators,
Holoscan
applications
can
read
video
frames
from
the
databus
(to
be
used
as
the
source
for
workflow
processing)
and
write
processed
results
back
to
the
databus
(for
consumption
by
another
component).

Combining
these
two
applications
demonstrates
the
sidecar
dataflow
using
three
processes:

1. One
   
   dds_video
   process
   captures
   frames
   from
   the
   camera
   sensor
   and
   publishes
   them
   to
   DDS.
2. A
   
   body_pose_estimation
   process
   receives
   the
   input
   sensor
   frames
   from
   DDS,
   processes
   the
   frames
   through
   the
   body
   pose
   estimation
   model,
   then
   outputs
   the
   frames
   with
   the
   inference
   results
   overlaid
   on
   top
   of
   the
   images
   to
   DDS.
3. Another
   
   dds_video
   process
   receives
   the
   processed
   frames
   and
   renders
   them
   to
   the
   display.

To
run
this
example
locally,
start
by
reading
the

HoloHub
DDS
Operators
documentation
regarding
the
dependency
requirements
for
setting
up
RTI
Connext.
To
learn
how
to
build
and
run
the
applications,
see
the

DDS
Support
section
of
the
Body
Pose
Estimation
documentation.

Summary

Integrating
NVIDIA
Holoscan
with
RTI
Connext
offers
Holoscan
developers
in
the
medical
device
industry
numerous
advantages
in
the
transition
to
AI-enhanced
systems
and
devices.
These
include
seamless
integration
with
distributed
healthcare
systems
with
minimal
overhead,
the
enhancement
of
legacy
systems
with
advanced
AI
algorithms,
and
more.

Image
source:
Shutterstock