Skip to content

Research

Light source estimation

This effort addresses the problem of determining the location, direction, intensity, and color of the illuminants in a given scene. The problem has a broad range of applications in augmented reality, robust robot perception, and general scene understanding. In our research, we model complex light interactions with a custom path-tracer, capturing the effects of both direct and indirect illumination. Using a physically-based light model not only improve light source estimation, but will play a critical role in future research in surface property estimation and geometry refinement, ultimately leading to more accurate and complete scene reconstruction systems.

Publications:

  • Mike Kasper, Nima Keivan, Gabe Sibley, Christoffer Heckman. Light Source Estimation in Synthetic Images. In European Conference on Computer Vision, Virtual/Augmented Reality for Visual Artificial Intelligence Workshop 2016.

Sponsors:

  • Toyota grant 33643/1/ECNS20952N: Robust Perception.

Referring Expressions for Object Localization

Understanding references to objects based on attributes, spatial relationships, and other descriptive language expands the capability of robots to locate unknown objects (zero-shot learning), find objects in cluttered scenes, and communicate uncertainty with human collaborators. We are collecting a new set of annotations, SUNspot, for the SUNRGB-d scene understanding dataset. Unlike other referring expression datasets, SUNspot will focus on graspable objects in interior scenes accompanied by the depth sensor data and full semantic segmentation from the SUNRGB-d dataset. Using SUNspot, we hope to develop a novel referring expressions system that will improve object localization for use in human-robot interaction.

Compass

Compass is a simultaneous localization and mapping (SLAM) pipeline with extensible frontend capability and an optimization backend based on Ceres solver.

Publications:

  • Fernando Nobre, Christoffer Heckman. International Symposium on Experimental Robotics 2016.
  • Fernando Nobre, Mike Kasper, Christoffer Heckman. IEEE International Conference on Robotics and Automation 2017.

Sponsors:

  • DARPA #N65236-16-1-1000. DSO Seedling: Ninja Cars.
  • Toyota grant 33643/1/ECNS20952N: Robust Perception.

Parkour Cars

This project aims to develop high fidelity real-time systems for perception, planning and control of agile vehicles in challenging terrain including jumps and loop-the-loops. The current research is focused on the local planning and control problem. Due to the difficulty of the maneuvers, the planning and control systems must consider the underlying physical model of the vehicle and terrain. This style of simulation-in-the-loop planning enables very accurate prediction and correction of the vehicle state, as well as the ability to learn precise attributes of the underlying physical model.

Publications:

Sponsors:

  • NSF #1646556. CPS: Synergy: Verified Control of Cooperative Autonomous Vehicles.
  • DARPA #N65236-16-1-1000. DSO Seedling: Ninja Cars.
  • Toyota grant 33643/1/ECNS20952N: Robust Perception.