An Improved Tracking-Learning-Detection Algorithm for Low Frame Rate

Subject Areas : Renewable energy

Hooman Moridvaisi ¹ , Farbod Razzazi ² , Mohammad Ali Pourmina ³ , Massoud Dousti ⁴

1 - Department of Electrical and Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
2 - Department of Electrical and Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
3 - Department of Electrical and Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran
4 - Department of Electrical and Computer Engineering, Science and Research Branch, Islamic Azad University, Tehran, Iran

Received: 2021-12-09 Accepted : 2022-02-24 Published : 2023-08-23

Keywords: target tracking, Machine learning algorithm, Mean-Shift algorithm, Low frame rate, Tracking learning detection,

Abstract :

The conventional Tracking-Learning-Detection (TLD) algorithm is sensitive to illumination change and clutter and low frame rate and results in drift even missing. To overcome these shortcomings and increase robustness, by improving the TLD structure via integrating mean-shift and co-training learning can be achieved better results undergo low frame rate (LFR) condition and the robustness and accuracy tracking of the TLD structure increases. Because of, the Mean-Shift tracking algorithm is robust to rotation, partial occlusion and scale changing and it is simple to implement and takes less computational time. On the other, the co-training learning algorithm with two independent classifiers can learn changes of the target features in during the online tracking process. Therefore, the extended structure can solve the problem of lost object tracking in LFR videos and other challenges simultaneously. Finally, comparative evaluations of the proposed method to other top state-of-the-art tracking algorithms under the various scenarios from the TB-100 known dataset, demonstrate the superior performance of the proposed algorithm compared to other tracking algorithms in terms of tracking robustness and stability performance. Finally, the proposed structure based on the TLD architecture, in scenarios with the various challenges mentioned, will improve on average about 33% of the results, compared to the traditional TLD algorithm.

References:

[1] G. Lee, R. Mallipeddi, M. Lee, “Trajectory-based vehicle tracking at low frame rates”, Expert Systems with Applications, vol. 80, pp. 46-57, Sept. 2017 (doi:10.1016/j.eswa.2017.03.023).

[2] L. Liu, J. Cao, “End-to-end learning interpolation for object tracking in low frame-rate video”, IET Image Processing, vol. 14, no. 6, pp. 1066-1072, May 2020 (doi:10.1049/iet-ipr.2019.0944).

[3] X. Zhang, W. Hu, N. Xie, H. Bao, S. Maybank, "A robust tracking system for low frame rate video", International Journal of Computer Vision, vol. 115, no. 3, pp. 279-304, Sept. 2015 (doi: 10.1007/s11263-015-0819-8).

[4] W. Zhong, H. Lu, M.H. Yang, "Robust object tracking via sparsity-based collaborative model", Proceeding of the IEEE/CVPR, pp. 1838-1845, Providence, RI, USA, June 2012 (doi:10.1109/CVPR.2012.6247882).

[5] Y. Wu, B. Shen, H. Ling, "Online robust image alignment via iterative convex optimization", Proceeding of the IEEE/CVPR, pp. 1808-1814, Providence, RI, USA, June 2012 (doi:10.1109/CVPR.2012.6247878).

[6] L. Sevilla-Lara, E. Learned-Miller, "Distribution fields for tracking", Proceeding of the IEEE/CVPR, pp. 1910-1917, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2012.6247891).

[7] C. Bao, Y. Wu, H. Ling, H. Ji,"Real time robust l1 tracker using accelerated proximal gradient approach", Proceeding of the IEEE/CVPR, pp. 1830-1837, Providence, RI, USA, June 2012 (doi: 10.1109/CVPR.2­012­.62­47881).

[8] S. Oron, A. Bar-Hillel, D. Levi, S. Avidan, "Locally orderless tracking", International Journal of Computer Vision, vol. 111, no. 2, pp. 213-228, Sept. 2015 (doi:10.1007/s11263-014-0740-6).

[9] J.F. Henriques, R. Caseiro, P. Martins, J. Batista, "Exploiting the circulant structure of tracking-by-detection with kernels", Proceeding of the ECCV, pp. 702-715, Berlin, Heidelberg, Oct. 2012 (doi:10.1007/978-3-642-33765-9-50).

[10] J.F. Henriques, R. Caseiro, P. Martins, J. Batista, “High-speed tracking with kernelized correlation filters”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 37, no. 3, pp.583-596, March 2015 (doi: 10.1109/TPAMI.2014.2345390).

[11] X. Lu, C. Ma, B. Ni, X. Yang, “Adaptive region proposal with channel regularization for robust object tracking”, IEEE Trans. on Circuits and Systems for Video Technology, vol. 31, no. 9, pp. 1268-1282, April 2021 (doi: 10.1109/TCSVT.2019.2944654).

[12] F. Porikli, O. Tuzel, “Object tracking in low-frame-rate video”, Proceeding of the SPIE, pp. 1-8, Cambridge, USA, Mar. 2005 (doi:10.1117/12.587907).

[13] Y. Li, H. Ai, T. Yamashita, S. Lao, M. Kawade,“Tracking in low frame rate video: ACascade particle filter with discriminative observers of different life spans”, IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1728-1740, Oct. 2008 (doi:10.1109/TPAMI.2008.73).

[14] Z. Li, J. Chen, N.N. Schraudolph, “An improved meanshift tracker with kernel prediction and scale optimisation targeting for low-frame-rate video tracking”, Proceeding of the IEEE/ICPR, pp.1-4, Tampa, FL, Dec. 2008 (doi: 10.1109/ICPR.2008.4761311).

[15] T. Zhang, S.. Fei, H. Lu, X. Li,”Modified particle filter for object tracking in low frame rate video”, Proceeding of the IEEE/CDC, Shanghai, China, pp. 2552-2557, Dec. 2009 (doi:10.1109/CDC.2009.5400892).

[16] M. Godec, P.M. Roth, H. Bischof, “Hough-based tracking of non-rigid objects”, Proceeding of the IEEE/CVIU, vol. 117, no. 10, pp. 1245-1256, Oct. 2013 (doi:10.1016/j.cviu.2012.11.005).

[17] H.S. Boroujeni, N.M. Charkari, M. Behrouzifar, “Tracking multiple variable-sizes moving objects in LFR videos using a novel genetic algorithm approach”, Knowledge Technology, Communications in Computer and Information Science, vol. 295, pp. 143-153 , July 2011 (doi:10.1007/978-3-642-32826-8-15).

[18] K. Palaniappan1, F. Bunyak, P. Kumar, I. Ersoy,S. Jaeger, K. Ganguli, A. Haridas, J. Fraser,R. M. Rao, G. Seetharaman,”Efficient feature extraction and likelihood fusion for vehicle tracking in low frame rate airborne video”, Proceeding of the IEEE/FUSION, pp. 1-8,  Edinburgh, UK, Feb. 2010 (doi: 10.1109/ICIF.201­0.571­18­91).

[19]  Y. Pang, D. Shenb, G. Chen, P. Liang, K. Pham, E. Blasch, Z. Wang, H. Ling, ”Low frame rate video target localization and tracking testbed”, Proceeding of the SPIE, vol. 8742, pp. 1-6, May 2013 (doi: 10.1117/12.20­15­954).

[20] G. Lee, R. Mallipeddi, M. Lee, “Trajectory-based vehicle tracking at low frame rates”, Expert Systems with Applications, vol. 80, pp. 46-57, Sept.  2017 (doi: 10.1016/j.eswa.2017.03.023).

[21] W. Dai, T. Chang, K. Su, Q. Wang, "Improved TLD target algorithm based on feature fusion", Proceeding of the WARTIA, pp. 398-403, May 2016 (doi: 10.2991/wartia-16.20­16.7­9).

[22] T. Xu, C. Huang, Q. He, G. Guan, Y. Zhang, "An improved TLD target tracking algorithm", Proceeding of the IEEE/ICIA, vol. 10033, pp. 2051-2055, Ningbo, China, Aug. 2016 (doi:10.1109/ICInfA.2016.7832157).

[23]  L. Yu, T. Zheng, Q. Shi, "Image tracking algorithm improvement based on TLD frame", International Journal of Signal Processing, Image Processing and Pattern Recognition, vol 9, no. 5, pp. 431-440, May 2016 (doi: 10.14­257/ij­sip.2016.9.5.38).

[24]  T. Li, W. J. Zhao, S. Yang, C. Li, "An improved TLD object tracking algorithm", Proceeding of the ICDIP, vol 9, no. 5, pp. 2051-2055, Aug. 2016 (doi:10.1117/12.2244919).

[25]  L. Zhao, Y. Chen, Q. Ye, "An improved TLD algorithm based on Kalman filter and SURF feature matching", Proceeding of the AIP, vol. 1839, no. 1, pp. 1-6, May 2017 (doi:10.1063/1.4982579).

[26]  J. Hu, M. Cai, J. Li, "An improved TLD method based on color feature", Proceeding of the IEEE/CCDC, pp. 6096-6101, Chinese, May 2017 (doi: 10.1109/CCDC.2017.7978266).

[27] Z. Song, Z. Cong, Z. Yanan, D. Yuren, "An improved TLD target tracking algorithm based on Mean Shift", Proceeding of the IEEE/ICEMI, pp. 387-391, Yangzhou, China, Oct. 2017 (doi: 10.1109/ICEMI.­2017.8­265­8­27).

[28] J. Zhang, A. Wang, M. Wang, Y. Iwahori, "A novel target algorithm based on TLD combining with SLBP", International Journal of Performability Engineering, vol. 13, no. 4, pp. 458-468, July 2017 (doi: 10.23940/ij­pe.17.04.p13.458468).

[29] E. Dong, M. Deng, J. Tong, C. Jia, S. Du,  “Moving vehicle tracking based on improved tracking–learning–detection algorithm”, IET Computer Vision, vol. 13, no. 8, pp. 730-741, Dec. 2019 (doi: 10.1049/iet-cvi.2­018.5­787).

[30] Y. Zhu, C. Wang, Y. Niu, L. Wu, “hTLD: A Human-in-the-loop target detection and tracking method for UAV”, Proceeding of the IEEE/CGNCC), pp. 1-6, Xiamen, China, Aug. 2018 (doi: 10.1109/GNC­C429­60.2­01­8.­9018702).

[31] X. Yang, S. Zhu, S. Xia, D. Zhou, “A new TLD target tracking method based on improved correlation filter and adaptive scale”, The Visual Computer, vol. 36, no. 9, pp.1783-1795, Sept. 2020 (doi: 10.1007/s00371-019-01772-w).

[32] H. Moridvaisi, F. Razzazi, M.A. Pourmina, M. Dousti, “An extended KCF tracking algorithm based on TLD structure in low frame rate videos”, Multimedia Tools and Applications, vol. 79, no. 29, pp. 20995-21012, Aug. 2020 (doi: 10.1007/s11042-020-08867-w).

[33] J. Wang, M. Zhao, L. Zou, Y. Hu, X. Cheng, X. Liu, “Fish tracking based on improved TLD algorithm in real-world underwater environment”, Marine Technology Society Journal, vol. 53, no. 3, pp. 80-89, May 2019 (doi: 10.4031/MTSJ.53.3.8).

[34] X. Zhen, S. Fei, Y. Wang, W. Du, “A visual object tracking algorithm based on improved TLD”, Algorithms, vol. 13, no. 1, Article Number: 15, Jan. 2020 (doi:10.3390/a13010015).

[35] L. Zhang, J. Hou, M. Chen, H. Li, “The improved TLD algorithm that combines CAMShift and orientation prediction to realize face tracking in low illumination”, Proceeding of the IEEE/CAC, pp. 3193-3200, Chinese, Nov. 2020 (doi: 10.1109/CAC51589.2020.9326983).

36] Q. Ding, Z. Ding, “Machine learning model for feature recognition of sports competition based on improved TLD algorithm”, Journal of Intelligent & Fuzzy Systems, vol. 4, no. 1, pp. 1-12, Jan. 2021 (doi: 10.3233/JIFS-189312).

[37] Z. Kalal, K. Mikolajczyk, J. Matas, “Tracking-learning-detection”,  IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1409-1422, Dec. 2011 (doi: 10.1109/TPAMI.2011.239).

[38] K. Nummiaro, E. Koller-Meier, L.J.V. Gool, “An adaptive color-based particle filter”, Image Vision Computer, vol. 21, no. 1, pp. 99–110, Dec. 2003 (doi: 10.1016/S0262-8856(02)00129-4).