• Document: Geometric Models for Rolling-shutter and Push-broom Sensors. Erik Ringaby
  • Size: 4.3 MB
  • Uploaded: 2019-05-24 01:04:42
  • Status: Successfully converted

Some snippets from your converted document:

Linköping Studies in Science and Technology Dissertation No. 1615 Geometric Models for Rolling-shutter and Push-broom Sensors Erik Ringaby Department of Electrical Engineering Linköping University, SE-581 83 Linköping, Sweden Linköping August 2014 Geometric Models for Rolling-shutter and Push-broom Sensors c 2014 Erik Ringaby Department of Electrical Engineering Linköping University SE-581 83 Linköping Sweden ISBN 978-91-7519-255-0 ISSN 0345-7524 Linköping Studies in Science and Technology Dissertation No. 1615 iii Abstract Almost all cell-phones and camcorders sold today are equipped with a CMOS (Complementary Metal Oxide Semiconductor) image sensor and there is also a general trend to incorporate CMOS sensors in other types of cameras. The CMOS sensor has many advantages over the more conventional CCD (Charge-Coupled Device) sensor such as lower power consumption, cheaper manufacturing and the potential for on-chip processing. Nearly all CMOS sensors make use of what is called a rolling shutter readout. Unlike a global shutter readout, which images all the pixels at the same time, a rolling-shutter exposes the image row-by-row. If a mechanical shutter is not used this will lead to geometric distortions in the image when either the camera or the objects in the scene are moving. Smaller cameras, like those in cell-phones, do not have mechanical shutters and systems that do have them will not use them when recording video. The result will look wobbly (jello effect), skewed or otherwise strange and this is often not desirable. In addition, many computer vision algorithms assume that the camera used has a global shutter and will break down if the distortions are too severe. In airborne remote sensing it is common to use push-broom sensors. These sensors exhibit a similar kind of distortion as that of a rolling-shutter camera, due to the motion of the aircraft. If the acquired images are to be registered to maps or other images, the distortions need to be suppressed. The main contributions in this thesis are the development of the three-dimen- sional models for rolling-shutter distortion correction. Previous attempts modelled the distortions as taking place in the image plane, and we have shown that our techniques give better results for hand-held camera motions. The basic idea is to estimate the camera motion, not only between frames, but also the motion during frame capture. The motion is estimated using image correspondences and with these a non-linear optimisation problem is formulated and solved. All rows in the rolling-shutter image are imaged at different times, and when the motion is known, each row can be transformed to its rectified position. The same is true when using depth sensors such as the Microsoft Kinect, and the thesis describes how to estimate its 3D motion and how to rectify 3D point clouds. In the thesis it has also been explored how to use similar techniques as for the rolling-shutter case, to correct push-broom images. When a transformation has been found, the images need to be resampled to a regular grid in order to be visualised. This can be done in many ways and different methods have been tested and adapted to the push-broom setup. In addition to rolling-shutter distortions, hand-held footage often has shaky camera motion. It is possible to do efficient video stabilisation in combination with the rectification using rotation smoothing. Apart from these distortions, motion blur is a big problem for hand-held photography. The images will be blurry due to the camera motion and also noisy if taken in low light conditions. One of the contributions in the thesis is a method which uses gyroscope measurements and feature tracking to combine several images, taken with a smartphone, into one resulting image with less blur and noise. This enables the user to take photos which would have otherwise required a tripod. iv v Populärvetenskaplig sammanfattning Nästan alla mobiltelefoner och videokameror som säljs idag är utrustade med en CMOS-bildsensor (Complementary Metal Oxide Semiconductor) och det finns även en allmän trend att använda CMOS-sensorer i andra typer av kameror. Sensorn har många fördelar jämfört med den mer konventionella CCD-sensorn (Charge-Coupled Device) såsom lägre strömförbrukning, billigare tillverkning och möjligheten att utföra beräkningar på chippet. CMOS-sensorer i konsumentpro- dukter använder sig av vad som kallas en rullande slutare. Till skillnad från en global slutare, där alla pixlar avbildas samtidigt, så exponerar en sensor med rul- lande slutare bilden rad fo

Recently converted files (publicly available):