Inferring the 3D World from Incomplete Observations: Representations and Data Priors
Jan Eric Lenssen
MPI-INF - D2
05 Jul 2023, 12:15 pm - 1:15 pm
Saarbrücken building E1 5, room 002
Joint Lecture Series
Computer Vision has become increasingly capable of representing the 3D world,
given large sets of dense and complete observations, such as images or 3D
scans. However, in comparison with humans, we still lack an important ability:
deriving 3D representations from just a few, incomplete 2D observations.
Replicating this ability might be the next important step towards a general
vision system. A key aspect of the human abilities is that observations are
complemented by previously learned information: the world is not only sensed -
to a large degree it is inferred. ...
Computer Vision has become increasingly capable of representing the 3D world,
given large sets of dense and complete observations, such as images or 3D
scans. However, in comparison with humans, we still lack an important ability:
deriving 3D representations from just a few, incomplete 2D observations.
Replicating this ability might be the next important step towards a general
vision system. A key aspect of the human abilities is that observations are
complemented by previously learned information: the world is not only sensed -
to a large degree it is inferred. The common way to approach this task with
deep learning are data priors, which capture information present in large
datasets and which are used to perform inference from novel observations.
This talk will discuss 3D data priors and their important connection to 3D
representations. Choosing the right representation, we can have abstract
control over which information is learned from data and how we can use it
during inference, which leads to more effective solutions than simply learning
everything end-to-end. Thus, the focus of my research and this talk will be on
representations with important properties, such as data efficiency and useful
equi- and invariances, which enable the formulation of sophisticated,
task-specific data priors. These presented concepts are showcased on examples
from my and collaborating groups, e.g., as data priors for reconstructing
objects or object interaction sequences from incomplete observations.
Read more