Fast Reconstruction of the World from Photos and Videos

In recent years photo/video sharing web sites like Flickr and YouTube have become increasingly popular. Nowadays, every day terra bytes of photos and videos are uploaded. These data survey large parts of the world throughout the di_erent seasons, various weather conditions and all times of the day. In the talk I will present my work on the highly e_cient reconstruction of 3D models from these data. It addresses a variety of the current challenges that have to be addressed to achieve a concurrent 3D model from these data. The challenges are: estimation of the geometric and radiometric camera calibration from videos and photos, e_cient robust camera motion estimation for (quasi-)degenerate estimation problems, high performance stereo estimation from multiple views, automatic selection of correct views from noisy image/video collections, image based location recognition for topology detection. In the talk I will discuss the details of our real-time camera motion estimation from video using our Adaptive Real-Time Random Sample Consensus (ARRSAC) and our high performance salient feature tracker, which simultaneously estimates the radiometric camera calibration and tracks the motion of the salient feature points. Furthermore our technique to achieve robustness against (quasi-) degenerate data will be introduced. It allows to detect and overcome the case of data, which under-constrain the camera motion estimation problem. Additionally our optimal stereo technique for determining the scene depths with constant precision throughout the scene volume will be explained during the talk. It allows to perform the scene depth estimation from a large set of views with optimal computational e_ort while obtaining the depth with constant precision throughout the reconstruction volume. I also discuss our fast technique for the image based location recognition, which uses commodity graphics processors to achieve real-time performance while providing high recognition rates. Furthermore in the talk I present our work on 3D reconstruction from internet photo collections. It combines image based recognition with geometric constraints to e_ciently perform the simultaneous selection of correct views and the 3D reconstruction from large collections of photos. The talk will also explain the future challenges in all the mentioned areas. Jan-Michael Frahm is a Research Assistant Professor at University of North Carolina at Chapel Hill. He received his Ph.D in computer vision in 2005 from the Christian-Albrechts University of Kiel, Germany. His Diploma in Computer Science is from the University of Lubeck. Dr.- Ing. Frahm`s research interests include a variety of computer vision problems. He has worked on structure from motion for single/multi-camera systems for static and dynamic scenes to create 3D models of the scene; real-time multi-view stereo to create a dense scene geometry from camera images; use of camera-sensor systems for 3D scene reconstruction with fusion of multiple orthogonal sensors; improved robust and fast estimation methods from noisy data to compensate for highly noisy measurements in various stages of the reconstruction process; high performance feature tracking for salient image-point motion extraction; and the development of data-parallel algorithms for commodity graphics hardware for e_cient 3D reconstruction.