Abstract
Person re-identification (Re-ID) has rapidly advanced due to its widespread real-world applications. It poses a significant risk of exposing private data from its training dataset. This paper aims to quantify this risk by conducting a membership inference (MI) attack. Most existing MI attack methods focus on classification models, while Re-ID follows a distinct paradigm for training and inference. Re-ID is a fine-grained recognition task that involves complex feature embedding, and the model outputs commonly used by existing MI algorithms, such as logits and losses, are inaccessible during inference. Since Re-ID models the relative relationship between image pairs rather than individual semantics, we conduct a formal and empirical analysis that demonstrates that the distribution shift of the inter-sample similarity between the training and test sets is a crucial factor for membership inference and exists in most Re-ID datasets and models. Thus, we propose a novel MI attack method based on the distribution of inter-sample similarity, which involves sampling a set of anchor images to represent the similarity distribution that is conditioned on a target image. Next, we consider two attack scenarios based on information that the attacker has. In the “one-to-one” scenario, where the attacker has access to the target Re-ID model and dataset, we propose an anchor selector module to select anchors accurately representing the similarity distribution. Conversely, in the “one-to-any” scenario, which resembles real-world applications where the attacker has no access to the target Re-ID model and dataset, leading to the domain-shift problem, we propose two alignment strategies. Moreover, we introduce the patch-attention module as a replacement for the anchor selector. Experimental evaluations demonstrate the effectiveness of our proposed approaches in Re-ID tasks in both attack scenarios.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
The datasets used during and analyzed during the current study are available in the following public domain resources: https://www.cs.toronto.edu/kriz/cifar.html; https://zheng-lab.cecs.anu.edu.au/Project/project_reid.html; https://www.pkuvmc.com/dataset.html; The models and source data generated during and analyzed during the current study are available from the corresponding author upon reasonable request.
References
Bousmalis, K., Silberman, N., Dohan, D., et al. (2017). Unsupervised pixel-level domain adaptation with generative adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3722–3731).
Chen, B., Deng, W., & Hu, J. (2019). Mixed high-order attention network for person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 371–381).
Chen, M., Zhang, Z., Wang, T., et al. (2021). When machine unlearning jeopardizes privacy. In Proceedings of the 2021 ACM SIGSAC conference on computer and communications security (pp. 896–911).
Cheng, D., Gong, Y., Zhou, S., et al. (2016). Person re-identification by multi-channel parts-based CNN with improved triplet loss function. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1335–1344).
Chollet, F. (2017). Xception: Deep learning with depthwise separable convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1251–1258).
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al. (2020). An image is worth 16x16 words: Transformers for image recognition at scale. arXiv:2010.11929
Duan, Y., Lu, J., Feng, J., et al. (2017). Deep localized metric learning. IEEE Transactions on Circuits and Systems for Video Technology, 28(10), 2644–2656.
Fredrikson, M., Jha, S., & Ristenpart, T. (2015). Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security (pp. 1322–1333).
Ganin, Y., Ustinova, E., Ajakan, H., et al. (2016). Domain-adversarial training of neural networks. The Journal of Machine Learning Research, 17(1), 1–35.
Gao, J., Jiang, X., Zhang, H., et al. (2023). Similarity distribution based membership inference attack on person re-identification. In Proceedings of the AAAI conference on artificial intelligence (pp. 14,820–14,828).
Gong, B., Grauman, K., & Sha, F. (2014). Learning kernels for unsupervised domain adaptation with applications to visual object recognition. International Journal of Computer Vision, 109(1), 3–27.
Hayes, J., Melis, L., Danezis, G., et al. (2017). Logan: Membership inference attacks against generative models. arXiv:1705.07663
He, K., Zhang, X., Ren, S., et al. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778).
He, S., Luo, H., Wang, P., et al. (2021). Transreid: Transformer-based object re-identification. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 15,013–15,022).
Hu, H. M., Fang, W., Zeng, G., et al. (2017). A person re-identification algorithm based on pyramid color topology feature. Multimedia Tools and Applications, 76(24), 26,633-26,646.
Krizhevsky, A. (2009). Learning multiple layers of features from tiny images. Master’s Thesis, University of Tront.
Li, G., Rezaei, S., & Liu, X. (2022). User-level membership inference attack against metric embedding learning. arXiv:2203.02077
Li, J., Li, N., & Ribeiro, B. (2020). Membership inference attacks and defenses in supervised learning via generalization gap. arXiv:2002.12062.
Li, W., Zhu, X., & Gong, S. (2018). Harmonious attention network for person re-identification. In 2018 IEEE/CVF conference on computer vision and pattern recognition (pp. 2285–2294).
Liu, W., Wen, Y., Yu, Z., et al. (2016). Large-margin softmax loss for convolutional neural networks. In Proceedings of the 33rd international conference on international conference on machine learning—Volume 48. JMLR.org, ICML’16 (pp. 507–516).
Liu, Z., Lin, Y., Cao, Y., et al. (2021). Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 10,012–10,022).
Long, Y., Bindschaedler, V., Wang, L., et al. (2018). Understanding membership inferences on well-generalized learning models. arXiv:1802.04889
Ming, Z., Zhu, M., Wang, X., et al. (2022). Deep learning-based person re-identification methods: A survey and outlook of recent works. Image and Vision Computing, 119(104), 394.
Nasr, M., Shokri, R., & Houmansadr, A. (2018a). Comprehensive privacy analysis of deep learning. In Proceedings of the 2019 IEEE symposium on security and privacy (SP) (pp. 1–15).
Nasr, M., Shokri, R., & Houmansadr, A. (2018b). Machine learning with membership privacy using adversarial regularization. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security (pp. 634–646).
Oh Song, H., Xiang, Y., Jegelka, S., et al (2016). Deep metric learning via lifted structured feature embedding. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4004–4012).
Ranjan, R., Castillo, C. D., & Chellappa, R. (2017). L2-constrained softmax loss for discriminative face verification. arXiv:1703.09507
Sablayrolles, A., Douze, M., Schmid, C., et al. (2019). White-box vs black-box: Bayes optimal strategies for membership inference. In International conference on machine learning, PMLR (pp. 5558–5567).
Salem, A., Zhang, Y., Humbert, M., et al. (2018). Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv:1806.01246
Sandler, M., Howard, A., Zhu, M., et al. (2018). Mobilenetv2: Inverted residuals and linear bottlenecks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 4510–4520).
Schroff, F., Kalenichenko, D., & Philbin, J. (2015). Facenet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 815–823).
Sharma, C., Kapil, S. R., & Chapman, D. (2021). Person re-identification with a locally aware transformer. arXiv:2106.03720
Shokri, R., Stronati, M., Song, C., et al. (2017). Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP) (pp. 3–18). IEEE.
Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Song, L., Shokri, R., & Mittal, P. (2019). Privacy risks of securing machine learning models against adversarial examples. In Proceedings of the 2019 ACM SIGSAC conference on computer and communications security (pp. 241–257).
Sun, Y., Zheng, L., Yang, Y., et al. (2018). Beyond part models: Person retrieval with refined part pooling (and a strong convolutional baseline). In Proceedings of the European conference on computer vision (ECCV) (pp. 480–496).
Szegedy, C., Liu, W., Jia, Y., et al. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp 1–9).
Wang, G., Yuan, Y., Chen, X., et al. (2018a). Learning discriminative features with multiple granularities for person re-identification. In Proceedings of the 26th ACM international conference on multimedia (pp. 274–282).
Wang, H., Zhu, X., Gong, S., et al. (2018b). Person re-identification in identity regression space. International Journal of Computer Vision, 126, 1288–1310.
Wei, L., Zhang, S., Gao, W., et al. (2018). Person transfer gan to bridge domain gap for person re-identification. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 79–88).
Wu, X., Fredrikson, M., Jha, S., et al. (2016). A methodology for formalizing model-inversion attacks. In 2016 IEEE 29th computer security foundations symposium (CSF) (pp. 355–370). IEEE.
Yang, F., Yan, K., Lu, S., et al. (2019). Attention driven person re-identification. Pattern Recognition, 86, 143–155.
Yeom, S., Giacomelli, I., Fredrikson, M., et al. (2018). Privacy risk in machine learning: Analyzing the connection to overfitting. In2018 IEEE 31st computer security foundations symposium (CSF) (pp. 268–282). IEEE.
Yin, J., Wu, A., & Zheng, W. S. (2020). Fine-grained person re-identification. International Journal of Computer Vision, 128, 1654–1672.
Yu, D., Zhang, H., Chen, W., et al. (2021). How does data augmentation affect privacy in machine learning? In Proceedings of the AAAI conference on artificial intelligence (pp. 10,746–10,753).
Zhang, S., Chen, D., Yang, J., et al. (2021). Guided attention in CNNs for occluded pedestrian detection and re-identification. International Journal of Computer Vision, 129, 1875–1892.
Zheng, F., Deng, C., Sun, X., et al. (2019). Pyramidal person re-identification via multi-loss dynamic training. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8514–8522).
Zheng, L., Shen, L., Tian, L., et al. (2015). Scalable person re-identification: A benchmark. In Proceedings of the IEEE international conference on computer vision (pp. 1116–1124).
Zheng, L., Yang, Y., & Hauptmann, A. G. (2016). Person re-identification: Past, present and future. arXiv:1610.02984
Zheng, Z., Zheng, L., & Yang, Y. (2017). Unlabeled samples generated by gan improve the person re-identification baseline in vitro. In Proceedings of the IEEE international conference on computer vision (pp. 3754–3762).
Zhou, K., Yang, Y., Cavallaro, A., et al. (2019). Omni-scale feature learning for person re-identification. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 3702–3712).
Zhu, Z., Jiang, X., Zheng, F., et al. (2020). Aware loss with angular regularization for person re-identification. In Proceedings of the AAAI conference on artificial intelligence (pp. 13,114–13,121).
Acknowledgements
This work was supported by the National Natural Science Fund of China (62076184, 61976158, 61976160, 62076182, 62276190), in part by Fundamental Research Funds for the Central Universities and State Key Laboratory of Integrated Services Networks (Xidian University), in part by Shanghai Innovation Action Project of Science and Technology (20511100700) and Shanghai Natural Science Foundation (22ZR1466700).
Author information
Authors and Affiliations
Corresponding author
Additional information
Communicated by Segio Escalera.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Gao, J., Jiang, X., Dou, S. et al. Re-ID-leak: Membership Inference Attacks Against Person Re-identification. Int J Comput Vis (2024). https://doi.org/10.1007/s11263-024-02115-6
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s11263-024-02115-6