ISSN 0253-2778

CN 34-1054/N

open
Free to Publish. Free to Read.
Open AccessOpen Access JUSTC Original Paper

Differential privacy protection method for deep learning based on WGAN feedback

Funds:  Supported by the Key Research and Development Program Project of Anhui Province of China(201904d07020020), the Natural Science Foundation Project of Anhui Province of China(1908085MF212, 2008085MF190, 1808085QF210)， the Program for Synergy Innovation in the Anhui Higher Education Institutions of China(GXXT-2020-012).
Cite this:
https://doi.org/10.3969/j.issn.0253-2778.2020.08.004
More Information
  • Corresponding author: TAO Tao(corresponding author), male, born in 1977, PhD/Associate Professor.
  • Received Date: 05 June 2020
  • Accepted Date: 18 August 2020
  • Rev Recd Date: 18 August 2020
  • Publish Date: 31 August 2020
    Abstract

  • Abstract

    Aiming at the problem that attackers may steal sensitive information of the deep learning training dataset by some technological means such as the Generative Adversarial Network(GAN), combining the differential privacy theory, the differential privacy protection method was proposed for deep learning based on the Wasserstein generative adversarial network(WGAN) feedback parameter tuning. This privacy protection method is realized by optimization of the stochastic gradient descent, gradient clipping of setting gradient threshold, and noise adding to the optimization process of deep learning; WGAN was used to generate optimized results similar to the original data. The difference of the generated results and the original data were used for feedback parameter tuning. The experiment result shows that this method can effectively protect sensitive private information of the dataset and has preferable data usability.

    Abstract

    Aiming at the problem that attackers may steal sensitive information of the deep learning training dataset by some technological means such as the Generative Adversarial Network(GAN), combining the differential privacy theory, the differential privacy protection method was proposed for deep learning based on the Wasserstein generative adversarial network(WGAN) feedback parameter tuning. This privacy protection method is realized by optimization of the stochastic gradient descent, gradient clipping of setting gradient threshold, and noise adding to the optimization process of deep learning; WGAN was used to generate optimized results similar to the original data. The difference of the generated results and the original data were used for feedback parameter tuning. The experiment result shows that this method can effectively protect sensitive private information of the dataset and has preferable data usability.
    • FullText(HTML)
  • loading
    • References(14)
  • [1]
    HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
    [2]
    SWEENEY L. K-anonymity: A model for protecting privacy[J]. International Journal of Uncertainty: Fuzziness and Knowledge-Based Systems, 2002, 10(05):557-570.
    [3]
    MACHANAVAJJHALA A, KIFER D, GEHRKE J. L-diversity: Privacy beyond k -anonymity[J]. Acm Transactions on Knowledge Discovery from Data, 2007, 1(1): 3.
    [4]
    LI N, LI T, VENKATASUBRAMANIAN S. t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[C]//IEEE 23rd International Conference on Data Engineering. Piscataway: IEEE, 2007.
    [5]
    DWORK C. Differential privacy[J]. Lecture Notes in Computer Science, 2006, 26(2):1-12.
    [6]
    ABADI M, GOODFELLOW I, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// ACM Sigsac Conference on Computer & Communications Security. 2016.
    [7]
    PAPERNOT N, ABADI M, ERLINGSSON L, et al. Semi-supervised knowledge transfer for deep learning from private training data[C/OL]. (2017-03-03)[2020-05-05]. https://arxiv.org/pdf/1610.05755v4.
    [8]
    GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// International Conference on Neural Information Processing Systems. 2014.
    [9]
    RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. Computer Science, 2015.
    [10]
    ULLOA A, BASILE A, WEHNER G J, et al. An unsupervised homogenization pipeline for clustering similar patients using electronic health record data [C/OL]. (2018-03-21)[2020-05-05]. https://arxiv.org/pdf/1801.00065.
    [11]
    XIE L, LIN K, WANG S, et al. Differentially private generative adversarial network [C/OL]. (2018-02-19)[2020-05-05]. https://arxiv.org/pdf/1802.06739.
    [12]
    HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: Information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: Association for Computing Machinery, 2017: 603-618.
    [13]
    ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[C/OL]. (2017-12-06)[2020-05-05]. https://arxiv.org/pdf/1701.07875.
    [14]
    LECUN Y, CORTES C, BURGES C J. The MNIST database of handwritten digits[DB/OL].[2020-05-05]. http://yann.lecun.com/exdb/mnist/?o=3510.)
    • Supplements(0)
    • Related Articles
    • Track Citations
  • 加载中

Catalog

    |
    Get Citation
    PDF
    XML
    [1]
    HINTON G E, OSINDERO S, TEH Y W. A fast learning algorithm for deep belief nets[J]. Neural Computation, 2006, 18(7):1527-1554.
    [2]
    SWEENEY L. K-anonymity: A model for protecting privacy[J]. International Journal of Uncertainty: Fuzziness and Knowledge-Based Systems, 2002, 10(05):557-570.
    [3]
    MACHANAVAJJHALA A, KIFER D, GEHRKE J. L-diversity: Privacy beyond k -anonymity[J]. Acm Transactions on Knowledge Discovery from Data, 2007, 1(1): 3.
    [4]
    LI N, LI T, VENKATASUBRAMANIAN S. t-Closeness: Privacy Beyond k-Anonymity and l-Diversity[C]//IEEE 23rd International Conference on Data Engineering. Piscataway: IEEE, 2007.
    [5]
    DWORK C. Differential privacy[J]. Lecture Notes in Computer Science, 2006, 26(2):1-12.
    [6]
    ABADI M, GOODFELLOW I, GOODFELLOW I, et al. Deep Learning with Differential Privacy[C]// ACM Sigsac Conference on Computer & Communications Security. 2016.
    [7]
    PAPERNOT N, ABADI M, ERLINGSSON L, et al. Semi-supervised knowledge transfer for deep learning from private training data[C/OL]. (2017-03-03)[2020-05-05]. https://arxiv.org/pdf/1610.05755v4.
    [8]
    GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]// International Conference on Neural Information Processing Systems. 2014.
    [9]
    RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[J]. Computer Science, 2015.
    [10]
    ULLOA A, BASILE A, WEHNER G J, et al. An unsupervised homogenization pipeline for clustering similar patients using electronic health record data [C/OL]. (2018-03-21)[2020-05-05]. https://arxiv.org/pdf/1801.00065.
    [11]
    XIE L, LIN K, WANG S, et al. Differentially private generative adversarial network [C/OL]. (2018-02-19)[2020-05-05]. https://arxiv.org/pdf/1802.06739.
    [12]
    HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: Information leakage from collaborative deep learning[C]// Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. New York: Association for Computing Machinery, 2017: 603-618.
    [13]
    ARJOVSKY M, CHINTALA S, BOTTOU L. Wasserstein GAN[C/OL]. (2017-12-06)[2020-05-05]. https://arxiv.org/pdf/1701.07875.
    [14]
    LECUN Y, CORTES C, BURGES C J. The MNIST database of handwritten digits[DB/OL].[2020-05-05]. http://yann.lecun.com/exdb/mnist/?o=3510.)
    Recommended articles
    TrendMD
    Volume 50 Issue 8 page: 1064-1071
    Cover

    Article Metrics

    Article views (97) PDF downloads(126)
    Proportional views

    Copyright © Editorial Office of JUSTC, All Rights Reserved. 皖ICP备05002528号

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return