Human Behavior-Based Target Tracking With an Omni-Directional Thermal Camera

Benli E., Motai Y., Rogers J.

IEEE TRANSACTIONS ON COGNITIVE AND DEVELOPMENTAL SYSTEMS, vol.11, pp.36-50, 2019 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 11
  • Publication Date: 2019
  • Doi Number: 10.1109/tcds.2017.2726356
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus
  • Page Numbers: pp.36-50
  • Keywords: Behavior-based target tracking, far infrared (IR) camera, human behavior understanding, mobile robot, omnidirectional (O-D) camera, thermal vision, SHAPE REPRESENTATION, GAIT RECOGNITION, KALMAN FILTER, CURVE, INVARIANT, OBJECT, SYSTEM, MODEL, 3D
  • Karadeniz Technical University Affiliated: No


We investigate human behavior-based target tracking from omni-directional (O-D) thermal images for intelligent perception in unmanned systems. Current target tracking approaches arc primarily focused on perspective visual and infrared (IR) band, as well as O-D visual band tracking. The target tracking from O-D images and the use of O-D thermal vision have not been adequately addressed. Thermal O-D images provide a number of advantages over other passive sensor modalities such as illumination invariance, wide field-of-view, ease of identifying heat-emitting objects, and long term tracking without interruption. Unfortunately, thermal O-D sensors have not yet been widely used due to the following disadvantages: low resolution, low frame rates, high cost, sensor noise, and an increase in tracking time. This paper outlines a spectrum of approaches which mitigate these disadvantages to enable an O-D thermal IR camera equipped with a mobile robot to track a human in a variety of environments and conditions. The curve matched Kalman filter is used for tracking a human target based on the behavioral movement of the human and maximum a posteriori (MAP)-based estimation is extended for the human tracking as long term which provides a faster prediction. The benefits to using our MAP-based method are decreasing the prediction time of a target's position and increasing the accuracy of prediction of the next target position based on the target's previous behavior while increasing the tracking view and lighting conditions via the view from O-D IR camera.