Research Group 1546 (3D Tracks)

Mobile Augumented Reality System

In this project a mobile, camera and image-based system is to be developed that allows planners to compare, document and inspect multi-scale 3D building models and construction plans on-site and during the building phase. Beyond that the system is supposed to be used in the facility management to overlay 3D models with camera images of a mobile device.



Overview first period

During the first period of the project a camera system was materialized and calibrated. It is, together with a tablet PC, part of the mobile system that is supposed to allow planners to compare and visualize construction plans and alternatives on-site and in his office.

System design and calibration: At the beginning a prototype of a helmet camera system was assembled. It consisted of 3 fisheye cameras that were fixed to a wooden board. The radiating orientation guarantees a 360° coverage of the environment at all time and the amount of data is still processible. In order to project 3D models to the camera images, the relative orientation between the cameras was determined and a vertex shader was developed to project the 3D models geometrically correct into the fisheye camera mapping geometry.


  Figure 1: Top: new helmet camera system. Three fisheye cameras plus data processing unit fully integrated in one helmet. Bottom: prototype of camera system from the beginning of the project.  


Estimation of an initial camera pose within a textureless building model


One big challenge for all augmented reality applications is the robust estimation of an initial camera pose that can be used for the subsequent camera tracking. In outdoor applications this can be achieved by using the GPS/GNSS sensor of the mobile device to get a coarse but absolute estimate of the position. In close-range indoor environments this possibility does not exist and different methods have to be developed. Since we can rely on multi-scale building models in this research group we developed methods to directly use those to get an initial pose.



  Figure 2: Top row: Initialized camera pose. From here a pose refinement and the model-based tracking can commence. Bottom row: test environment in the building basement.  


The video shows how the particle filter initialization works. On the right side you can see the particles (pose hypothesis) and on the left side you can see initialized poses. Each time the model is reprojected to the cameras, the system is initialized. Until now there is no tracking involved.