logo

SCIENTIA SINICA Informationis, Volume 48, Issue 5: 545-563(2018) https://doi.org/10.1360/N112018-00017

Asymmetric person re-identification: cross-view person tracking in a large camera network

More info
  • ReceivedJan 17, 2018
  • AcceptedMar 27, 2018
  • PublishedMay 14, 2018

Abstract

Person re-identification (RE-ID) is critical and crucial for tracking people across multiple camera views, and hence, the piece-wise tracklets of each person from different locations can be connected. In this paper, we first review the development of person RE-ID and present its challenges. Subsequently, we introduce our recent development on asymmetric distance metric learning and the asymmetric person RE-ID modeling of the largely unsolved open-topic problems. Existing metric learning methods for person RE-ID usually ignore the characteristics of feature transformations between different camera views. The advantage of asymmetric metric is that it can model inconsistent feature transformations between different camera views. Except for being applied to general person RE-ID problem, asymmetric model can also be applied to cross-modality RE-ID, low-resolution RE-ID, attribute-image RE-ID, unsupervised RE-ID and partial RE-ID. Finally, we discuss the future development of person RE-ID.


Funded by

国家自然科学基金(61522115)


Acknowledgment

作者感谢如下几位同学的协助: 俞洪兴、叫洁宁、银舟、孟静珂.


References

[1] Collins R T, Lipton A J, Kanade T. Introduction to the special section on video surveillance. IEEE Trans Pattern Anal Mach Intel, 2000, 22: 745-746 CrossRef Google Scholar

[2] Xiang T, Gong S G. Video behavior profiling for anomaly detection. IEEE Trans Pattern Anal Mach Intel, 2008, 30: 893-908 CrossRef PubMed Google Scholar

[3] Wang L, Hu W M, Tan T N. A survey of visual analysis of human motion. Chin J Comput, 2002, 25: 225--237. Google Scholar

[4] Gheissari N, Sebastian T B, Hartley R. Person reidentification using spatiotemporal appearance. In: Proceedings of IEEE International Conference on Computer Vision and Pattern Recognition (CVPR), New York, 2006. 1528--1535. Google Scholar

[5] Gray D, Tao H. Viewpoint invariant pedestrian recognition with an ensemble of localized features. In: Proceedings of European Conference on Computer Vision (ECCV), Marseille, 2008. 262--275. Google Scholar

[6] Swain M J, Ballard D H. Color indexing. Int J Comput Vision, 1991, 7: 11-32 CrossRef Google Scholar

[7] Ojala T, Pietikainen M, Maenpaa T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans Pattern Anal Mach Intel, 2002, 24: 971-987 CrossRef Google Scholar

[8] Fogel I, Sagi D. Gabor filters as texture discriminator. Biol Cybern, 1989, 61: 103-113 CrossRef Google Scholar

[9] Wang X G, Doretto G, Sebastian T, et al. Shape and appearance context modeling. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Rio de Janeiro, 2007. 1--8. Google Scholar

[10] Farenzena M, Bazzani L, Perina A, et al. Person re-identification by symmetry-driven accumulation of local features. In: Proceedings of Computer Vision and Pattern Recognition (CVPR), San Francisco, 2010. 2360--2367. Google Scholar

[11] Ma B P, Su Y, Jurie F. Local descriptors encoded by fisher vectors for person re-identification. In: Proceedings of European Conference on Computer Vision Workshop (ECCV), Florence, 2012. 413--422. Google Scholar

[12] Kviatkovsky I, Adam A, Rivlin E. Color invariants for person reidentification. IEEE Trans Pattern Anal Mach Intel, 2013, 35: 1622-1634 CrossRef PubMed Google Scholar

[13] Ma B P, Su Y, Jurie F. Covariance descriptor based on bio-inspired features for person re-identification and face verification. Image Vision Comput, 2014, 32: 379-390 CrossRef Google Scholar

[14] Li W, Wang X G. Locally aligned feature transforms across views. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, 2013. 3594--3601. Google Scholar

[15] Layne R, Hospedales T M, Gong S G. Towards person identification and re-identification with attributes. In: Proceedings of European Conference on Computer Vision (ECCV), Florence, 2012. 402--412. Google Scholar

[16] Cheng D S, Cristani M, Stoppa M, et al. Custom pictorial structures for re-identification. In: Proceedings of British Machine Vision Conference (BMVC), Dundee, 2011. Google Scholar

[17] Wu Z Y, Li Y, Radke R J. Viewpoint invariant human re-identification in camera networks using pose priors and subject-discriminative features. IEEE Trans Pattern Anal Mach Intel, 2015, 37: 1095-1108 CrossRef PubMed Google Scholar

[18] Liao S C, Hu Y, Zhu X Y, et al. Person re-identification by local maximal occurrence representation and metric learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 2015. 2197--2206. Google Scholar

[19] Matsukawa T, Okabe T, Suzuki E, et al. Hierarchical gaussian descriptor for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1363--1372. Google Scholar

[20] Zhao R, Ouyang W L, Wang X G. Learning mid-level filters for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, 2014. 144--151. Google Scholar

[21] Chen T C, Zheng W S, Lai J H. Mirror representation for modeling view-specific transform in person re-identification. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), Buenos Aires, 2015. 3402--3408. Google Scholar

[22] Yang Y, Yang J M, Yan J J, et al. Salient color names for person re-identification. In: Proceedings of European Conference on Computer Vision (ECCV), Zurich, 2014. 536--551. Google Scholar

[23] Su C, Yang F, Zhang S L, et al. Multi-task learning with low rank attribute embedding for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 3739--3747. Google Scholar

[24] Su C, Zhang S L, Xing J L, et al. Deep attributes driven multi-camera person re-identification. In: Proceedings of European Conference on Computer Vision (ECCV), Amsterdam, 2016. 475--491. Google Scholar

[25] Li W, Zhao R, Xiao T, et al. DeepreID: deep filter pairing neural network for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, 2014. 152--159. Google Scholar

[26] Wu S X, Chen Y C, Li X, et al. An enhanced deep feature representation for person re-identification. In: Proceedings of IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Placid, 2016. 1--8. Google Scholar

[27] Ahmed E, Jones M, Marks T K. An improved deep learning architecture for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, 2015. 3908--3916. Google Scholar

[28] Xiao T, Li H S, Ouyang W L, et al. Learning deep feature representations with domain guided dropout for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1249--1258. Google Scholar

[29] Zhao L M, Li X, Wang J D, et al. Deeply-learned part-aligned representations for person re-identification. 2017,. arXiv Google Scholar

[30] Li D W, Chen X T, Zhang Z, et al. Learning deep context-aware features over body and latent parts for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2017. 384--393. Google Scholar

[31] Zhao H Y, Tian M Q, Sun S Y, et al. Spindle net: person re-identification with human body region guided feature decomposition and fusion. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 1077--1085. Google Scholar

[32] Barbosa I B, Cristani M, Bue A D, et al. Re-identification with RGB-D sensors. In: Proceedings of European Conference on Computer Vision (ECCV), Florence, 2012. 433--442. Google Scholar

[33] Munaro M, Basso A, Fossati A, et al. 3D reconstruction of freely moving persons for re-identification with a depth sensor. In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Hong Kong, 2014. 4512--4519. Google Scholar

[34] Takac B, Catala A, Rauterberg M, et al. People identification for domestic non-overlapping rgb-d camera networks. In: Proceedings of International Multi-Conference on Systems, Signals and Devices (SSD), Barcelona, 2014. Google Scholar

[35] Oliver J, Albiol A, Albiol A. 3D descriptor for people re-identification. In: Proceedings of International Conference on Pattern Recognition (ICPR), Tsukuba, 2012. 1395--1398. Google Scholar

[36] Haque A, Alahi A, Li F F. Recurrent attention models for depth-based person identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1229--1238. Google Scholar

[37] Wu A C, Zheng W S, Lai J H. Robust depth-based person re-identification. IEEE Trans Image Process, 2017, 26: 2588-2603 CrossRef PubMed ADS arXiv Google Scholar

[38] Ren L L, Lu J W, Feng J J. Multi-modal uniform deep learning for RGB-D person re-identification. Pattern Recogn, 2017, 72: 446-457 CrossRef Google Scholar

[39] Prosser B J, Zheng W S, Gong S G, et al. Person re-identification by support vector ranking. In: Proceedings of British Machine Vision Conference (BMVC), Aberystwyth, 2010. Google Scholar

[40] Zheng W S, Gong S G, Xiang T. Reidentification by relative distance comparison. IEEE Trans Pattern Anal Mach Intel, 2013, 35: 653-668 CrossRef PubMed Google Scholar

[41] Köstinger M, Hirzer M, Wohlhart P, et al. Large scale metric learning from equivalence constraints. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, 2012. 2288--2295. Google Scholar

[42] Tao D P, Jin L W, Wang Y F. Person re-identification by regularized smoothing kiss metric learning. IEEE Trans Circ Syst Video Technol, 2013, 23: 1675-1685 CrossRef Google Scholar

[43] Tao D P, Jin L W, Wang Y F. Person reidentification by minimum classification error-based KISS metric learning. IEEE Trans Cybernet, 2015, 45: 242-252 CrossRef PubMed Google Scholar

[44] Pedagadi S, Orwell J, Velastin S, et al. Local fisher discriminant analysis for pedestrian re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, 2013. 3318--3325. Google Scholar

[45] Mignon A, Jurie F. Pcca: a new approach for distance learning from sparse pairwise constraints. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, 2012. 2666--2672. Google Scholar

[46] Xu Y L, Lin L, Zheng W S, et al. Human re-identification by matching compositional template with cluster sampling. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, 2013. 3152--3159. Google Scholar

[47] Li Z, Chang S Y, Liang F, et al. Learning locally-adaptive decision functions for person verification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, 2013. 3610--3617. Google Scholar

[48] Ma L Y, Yang X K, Tao D C. Person re-identification over camera networks using multi-task distance metric learning. IEEE Trans Image Process, 2014, 23: 3656-3670 CrossRef PubMed ADS Google Scholar

[49] Lisanti G, Masi I, Bagdanov A D. Person re-identification by iterative re-weighted sparse ranking. IEEE Trans Pattern Anal Mach Intel, 2015, 37: 1629-1642 CrossRef PubMed Google Scholar

[50] Zhao R, Ouyang W L, Wang X G. Unsupervised salience learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, 2013. 3586--3593. Google Scholar

[51] Zhao R, Ouyang W L, Wang X G. Person re-identification by salience matching. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Sydney, 2013. 2528--2535. Google Scholar

[52] Zhao R, Oyang W L, Wang X G. Person re-identification by saliency learning. IEEE Trans Pattern Anal Mach Intel, 2017, 39: 356-370 CrossRef PubMed Google Scholar

[53] Liu X, Song M L, Tao D C, et al. Semi-supervised coupled dictionary learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, 2014.łinebreak 3550--3557. Google Scholar

[54] Xiong F, Gou M R, Camps O, et al. Person re-identification using kernel-based metric learning methods. In: Proceedings of European Conference on Computer Vision (ECCV), Zurich, 2014. 1--16. Google Scholar

[55] Yi D, Lei Z, Liao S C, et al. Deep metric learning for person re-identification. In: Proceedings of International Conference on Pattern Recognition (ICPR), Stockholm, 2014. 34--39. Google Scholar

[56] Karanam S, Li Y, Radke R J. Person re-identification with discriminatively trained viewpoint invariant dictionaries. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 4516--4524. Google Scholar

[57] Liao S C, Li S Z. Efficient PSD constrained asymmetric metric learning for person re-identification. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 3685--3693. Google Scholar

[58] Shen Y, Lin W Y, Yan J C, et al. Person re-identification with correspondence structure learning. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 3200--3208. Google Scholar

[59] Garcia J, Martinel N, Micheloni C, et al. Person re-identification ranking optimisation by discriminant context information analysis. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 1305--1313. Google Scholar

[60] Chen Y C, Zheng W S, Lai J H. An asymmetric distance model for cross-view feature mapping in person re-identification. IEEE Trans Circ Syst Video Technol, 2017, 27: 1661-1675 CrossRef Google Scholar

[61] Chen W H, Chen X T, Zhang J G, et al. Beyond triplet loss: a deep quadruplet network for person re-identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. Google Scholar

[62] Kodirov E, Xiang T, Fu Z Y, et al. Person re-identification by unsupervised $\ell_1$ graph learning. In: Proceedings of European Conference on Computer Vision (ECCV), Amsterdam, 2016. 178--195. Google Scholar

[63] Peng P X, Xiang T, Wang Y W, et al. Unsupervised cross-dataset transfer learning for person re-identification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1306--1315. Google Scholar

[64] You J J, Wu A C, Li X, et al. Top-push video-based person re-identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1345--1353. Google Scholar

[65] Wang T Q, Gong S G, Zhu X T, et al. Person re-identification by video ranking. In: Proceedings of European Conference on Computer Vision (ECCV), Zurich, 2014. 688--703. Google Scholar

[66] Wang T Q, Gong S G, Zhu X T. Person re-identification by discriminative selection in video ranking. IEEE Trans Pattern Anal Mach Intel, 2016, 38: 2501-2514 CrossRef PubMed Google Scholar

[67] McLaughlin N, Rincon J M, Miller P. Recurrent convolutional network for video-based person re-identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, 2016. 1325--1334. Google Scholar

[68] Zhu X K, Jing X Y, Wu F, et al. Video-based person re-identification by simultaneously learning intra-video and inter-video distance metrics. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), New York, 2016. 3552--3558. Google Scholar

[69] Zhou Z, Huang Y, Wang W, et al. See the forest for the trees: Joint spatial and temporal recurrent neural networks for video-based person re-identification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 6776--6785. Google Scholar

[70] Zhang W, Yu X D, He X Y. Learning bidirectional temporal cues for video-based person re-identification. IEEE Trans Circ Syst Video Technol, 2017, CrossRef Google Scholar

[71] Chung D, Tahboub K, Delp E J. A two stream siamese convolutional neural network for person re-identification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Venice, 2017. 1992--2000. Google Scholar

[72] Wang X J, Zheng W S, Li X. Cross-scenario transfer person re-identification. IEEE Trans Circuits Syst Video Technol, 2016, 26: 1447-1460 CrossRef Google Scholar

[73] Li W, Zhao R, Wang W G. Human reidentification with transferred metric learning. In: Proceedings of Asian Conference on Computer Vision (ACCV), Daejeon, 2012. 31--44. Google Scholar

[74] Ma A J, Yuen P C, Li J. Domain transfer support vector ranking for person re-identification without target camera label information. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Sydney, 2013. 3567--3574. Google Scholar

[75] Jiao J N, Zheng W S, Wu A C, et al. Deep low-resolution person re-identification. In: Proceedings of Association for the Advancement of Artificial Intelligence (AAAI), New Orleans, 2018. Google Scholar

[76] Li X, Zheng W S, Wang X J, et al. Multi-scale learning for low-resolution person re-identification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 3765--3773. Google Scholar

[77] Jing X Y, Zhu X, Wu F. Super-resolution person re-identification with semi-coupled low-rank discriminant dictionary learning. IEEE Trans Image Process, 2017, 26: 1363-1378 CrossRef PubMed ADS Google Scholar

[78] Wang Z, Hu R M, Yu Y, et al. Scale-adaptive low-resolution person re-identification via learning a discriminating surface. In: Proceedings of International Joint Conference on Artificial Intelligence (IJCAI), New York, 2016.łinebreak 2669--2675. Google Scholar

[79] Zheng W S, Gong S G, Xiang T. Associating groups of people. In: Proceedings of British Machine Vision Conference (BMVC), London, 2009. Google Scholar

[80] Zheng W S, Gong S, Xiang T. Transfer re-identification: from person to set-based verification. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, 2012. 2650--2657. Google Scholar

[81] Zheng W S, Gong S, Xiang T. Towards open-world person re-identification by one-shot group-based verification. IEEE Trans Pattern Anal Mach Intel, 2016, 38: 591-606 CrossRef PubMed Google Scholar

[82] Li S, Xiao T, Li H S, et al. Person search with natural language description. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 5187--5196. Google Scholar

[83] Xiao T, Li S, Wang B C, et al. Joint detection and identification feature learning for person search. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, 2017. 3376--3385. Google Scholar

[84] Chen Y C, Zheng W S, Lai J H. An asymmetric distance model for cross-view feature mapping in person re-identification. IEEE Trans Circ Syst Video Technol, 2017, 27: 1661-1675 CrossRef Google Scholar

[85] Chen Y C, Zhu X T, Zheng W S. Person re-identification by camera correlation aware feature augmentation. IEEE Trans Pattern Anal Mach Intel, 2018, 40: 392-408 CrossRef PubMed Google Scholar

[86] Zhu X, Wu B, Huang D. Fast open-world person re-identification. IEEE Trans Image Process, 2018, 27: 2286-2300 CrossRef PubMed ADS Google Scholar

[87] Wu A, Zheng W S, Yu H X, et al. Rgb-infrared cross-modality person re-identification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Venice, 2017. 5390--5399. Google Scholar

[88] Yin Z, Zheng W S, Wu A C, et al. Learning a semantically discriminative joint space for attribute based person re-identification. 2017,. arXiv Google Scholar

[89] Yu H X, Wu A, Zheng W S. Cross-view asymmetric metric learning for unsupervised person re-identification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Venice, 2017. 994--1002. Google Scholar

[90] Zheng W S, Li X, Xiang T, et al. Partial person re-identification. In: Proceedings of IEEE International Conference on Computer Vision (ICCV), Santiago, 2015. 4678--4686. Google Scholar

Copyright 2019 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有

京ICP备18024590号-1