F. Bunyak, K. Palaniappan, S. K. Nath, and G. Seetharaman

8th IEEE Workshop Applications of Computer Vision (WACV 2007), pgs. Online, 2007

motion, fusion, tracking, active contours, visual events, features

PlainText, Bibtex, PDF, DOI, Google Scholar

Abstract

Persistent object tracking in complex and adverse envi- ronments can be improved by fusing information from mul- tiple sensors and sources. We present a new moving object detection and tracking system that robustly fuses infrared and visible video within a level set framework. We also in- troduce the concept of the flux tensor as a generalization of the 3D structure tensor for fast and reliable motion detec- tion without eigen-decomposition. The infrared flux tensor provides a coarse segmentation that is less sensitive to illu- mination variations and shadows. The Beltrami color met- ric tensor is used to define a color edge stopping function that is fused with the infrared edge stopping function based on the grayscale structure tensor. The min fusion operator combines salient contours in either the visible or infrared video and drives the evolution of the multispectral geodesic active contour to refine the coarse initial flux tensor mo- tion blobs. Multiple objects are tracked using correspon- dence graphs and a cluster trajectory analysis module that resolves incorrect merge events caused by under- segmen- tation of neighboring objects or partial and full occlusions. Long-term trajectories for object clusters are estimated us- ing Kalman filtering and watershed segmentation. We have tested the persistent object tracking system for surveillance applications and demonstrate that fusion of visible and in- frared video leads to significant improvements for occlusion handling and disambiguating clustered groups of objects.