3D Reconstruction of Traffic Scenes
In traffic scenarios, where a number of cameras from different view angles is available,
object tracking and 3D reconstruction can be carried out.
In this context, we developed such an application that initially requires a fully calibrated camera system.
Camera images are transmitted from single servers for each camera to a central client, using image compression algorithms,
e.g. MPEG-4 or H.264. The camera sequences from all cameras from one scene are than used to provide information
for traffic control and additionally provide a 3D scene reconstruction.
The algorithm starts with a segmentation of moving scene objects from a static background.
The segmentation is based on a Kalman filter formalism that is able to adapt to environmental changes,
especially global illumination changes due to varying weather conditions.
After segmentation, a 3D scene model from the background images is constructed first by manually selecting
the appropriate ground area in each view. In the traffic environments this includes the street areas of the image.
Additional side areas can be selected further to model adjacent areas or buildings as perpendicular planes w.r.t. the ground plane.
The following example in Fig. 1 shows the modeled background image.
Interpolation between the views was achieved by assigning different plane vectors to each view
and is than handled automatically by today's graphics hardware.
By clicking on the image, the appropriate sequence should be opened in a suitable player.
Fig. 1: Background model of the traffic scene
Moving objects are first tracked within each 2D view separately.
During visibility an object ID or label is assigned to that object.
Afterwards the objects are merged in the 3D fusion stage.
By projecting the center of gravity of a 2D object contour into the other view,
the associated object in all other views is selected.
To provide a certain degree of error robustness due to possibly incorrect segmentation results,
the nearest object's center of gravity is chosen. Fig. 2 shows the label assignment after fusion,
where the same label number is associated with the same object in both views.
Fig. 2: Label assignment during the tracking process
In the final dynamic object reconstruction synthetic 3D models are selected from a database in a best-match approach.
The model is properly positioned according to the 3D center of gravity, which was obtained by projecting all
2D centers of gravity into the scene background using projection matrices.
Additionally the synthetic model is aligned in parallel to the motion trajectory and scaled to fit the original 2D object texture.
Finally object textures are mapped onto the model and the 3D model is integrated into the scene.
For further visual scene enhancement, trajectories of moving objects can be interpolated
for each render cycle instead of each camera frame rate.
The example in Fig. 3 shows the non-interpolated version with jerky object motion on the left and interpolated version on the right.
Fig. 3: Traffic scene with 3D object non-interpolated positions, i.e. temporal sparse positions according to camera frame rate left and
3D traffic scene with interpolated motion trajectories for moving objects, right.
Finally a 3D scene is reconstructed that allows free user navigation to better visualize traffic situations
and provide guidance by setting up a virtual camera path or flight through the 3D scene.
|
Start
Staff