Hybrid method for tracking moving objects in video streams under dynamic observation conditions

Abstract

A hybrid method for tracking objects in dynamic observation is proposed. Such conditions arise when the camera rotates and changes the zoom factor. The method uses the metadata of the camera position, including pitch, yaw, roll angles and zoom factors. The YOLO v8 artificial convolutional neural network was used as a detector. The main method of tracking the two is the stage of comparing detections and previous trajectories. At the first stage, the matching performed using the area overlap coefficient in pixel coordinates. At the second stage, the calculation of global coordinates is used based on the position of the object in the frame and the camera metadata. These global coordinates are using to predict the next position and compare with previous trajectories. The proposed method also allows determining the position of objects using data from altimeter sensors. The advantage of the proposed approach is the possibility of observation during sharp camera movements and changes in perspective. The method was experimentally tested on complex dynamic traffic scenes. The proposed method demonstrates higher metrics IDF1 = 0.84 and MOTA = 0.81 than standard algorithms on complex dynamic scenes. The method can be used in dynamic surveillance systems.