NVIDIA Introduces Deep-Learning Framework fVDB for Enhanced Spatial Intelligence

NVIDIA
has
unveiled
its
new
deep-learning
framework,
fVDB,
designed
to
build
spatial
intelligence
from
real-world
3D
data.
According
to
the

NVIDIA
Technical
Blog,
fVDB
aims
to
solve
the
inefficiencies
and
performance
bottlenecks
that
come
with
piecing
together
various
libraries
for
spatial
intelligence.

Challenges
in
Spatial
Intelligence

Generative
physical
AI
models
require
spatial
intelligence
to
understand
and
navigate
the
3D
space
of
the
physical
world.
Traditionally,
developers
have
had
to
use
a
patchwork
of
different
libraries
to
build
frameworks
for
spatial
intelligence,
leading
to
bugs,
inefficiencies,
and
performance
bottlenecks.

Introducing
fVDB

NVIDIA’s
fVDB
framework
is
designed
to
handle
sparse,
large-scale,
and
high-performance
spatial
intelligence.
Leveraging

OpenVDB,
an
industry-standard
for
the
efficient
storage
and
simulation
of
sparse
volumetric
data,
fVDB
integrates
deep
learning
operators
with

NanoVDB,
NVIDIA’s
GPU-accelerated
implementation
of
OpenVDB.

fVDB
is
an
open-source
extension
to

PyTorch,
enabling
a
complete
set
of
deep-learning
operations
on
large
3D
data.
Key
capabilities
include
compatibility
with
existing
VDB
datasets,
a
unified
API
for
neural
network
training,
ray
tracing,
and
rendering,
and
faster,
more
scalable
performance.

Applications
of
fVDB

fVDB
is
already
being
utilized
by
NVIDIA
Research,
NVIDIA
DRIVE,
and
NVIDIA
Omniverse
teams.
Notable
applications
include:

• 
  Surface
  Reconstruction:
  Neural
  Kernel
  Surface
  Reconstruction
  (NKSR)
  leverages
  fVDB
  to
  reconstruct
  high-fidelity
  surfaces
  from
  large
  point
  clouds.
• 
  Generative
  AI:
  XCube
  combines
  diffusion
  generative
  models
  with
  sparse
  voxel
  hierarchies,
  enabling
  the
  generation
  of
  3D
  scenes
  with
  high
  spatial
  resolution.
• 
  NeRFs:
  NeRF-XL
  uses
  fVDB
  to
  distribute
  neural
  radiance
  fields
  across
  multiple
  GPUs
  for
  large-scale
  3D
  rendering.

Future
Developments

NVIDIA
plans
to
integrate
fVDB
functionality
into
NVIDIA
NIM
microservices,
enabling
developers
to
incorporate
fVDB
into
Universal
Scene
Description
(OpenUSD)
workflows
within
NVIDIA
Omniverse.

Upcoming
NVIDIA
NIM
microservices
include
fVDB
Mesh
Generation,
fVDB
Physics
Super-Res,
and
fVDB
NeRF-XL,
which
will
generate
OpenUSD-based
geometry
using
Omniverse
Cloud
APIs.

Conclusion

Developed
by
NVIDIA,
fVDB
is
a
groundbreaking
deep-learning
framework
for
sparse,
large-scale
spatial
intelligence.
It
builds
on
OpenVDB
to
enable
applications
such
as
digital
twins,
neural
radiance
fields,
and
3D
generative
AI.

For
more
details,
visit
the

official
NVIDIA
announcement.

Image
source:
Shutterstock