ISSN 0253-2778

CN 34-1054/N

open
Free to Publish. Free to Read.
Open AccessOpen Access JUSTC Original Paper

Face color transfer based on optimal transport model

  • School of Mathematical Sciences, University of Science and Technology of China, Hefei 230026, China

Cite this:
https://doi.org/10.3969/j.issn.0253-2778.2018.02.009
  • Received Date: 09 January 2017
  • Rev Recd Date: 28 March 2017
  • Publish Date: 28 February 2018
    Abstract

  • Abstract

    Existing face color transfer algorithms are only based on Lαβ color space for matching mean and variance, and such transfer is limited to linear transformation and is usually applied to natural images. At the same time, these algorithms usually use facial landmark information to locate facial feature positions. However, the facial feature information thus obtained lacks accuracy and needs further optimization. To address these problems, a new face color transformation method was proposed based on the optimal transfer model to get the natural face color transfer effect. Firstly, the semantic information of the face was obtained directly by using the fully convolution network, and the corresponding face color transfer result was obtained by using the optimal transfer model. Experimental results show that the proposed algorithm is significantly improved in both the robustness of facial features segmentation and the subjective vision of face image color transfer.

    Abstract

    Existing face color transfer algorithms are only based on Lαβ color space for matching mean and variance, and such transfer is limited to linear transformation and is usually applied to natural images. At the same time, these algorithms usually use facial landmark information to locate facial feature positions. However, the facial feature information thus obtained lacks accuracy and needs further optimization. To address these problems, a new face color transformation method was proposed based on the optimal transfer model to get the natural face color transfer effect. Firstly, the semantic information of the face was obtained directly by using the fully convolution network, and the corresponding face color transfer result was obtained by using the optimal transfer model. Experimental results show that the proposed algorithm is significantly improved in both the robustness of facial features segmentation and the subjective vision of face image color transfer.
    • FullText(HTML)
  • loading
    • References(18)
  • [1]
    SHIRLEY P, ADHIKHMIN M, GOOCH B, et al. Color transfer between images[J]. IEEE Computer Graphics and Applications, 2001, 21: 34-41.
    [2]
    WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 277-280.
    [3]
    LEVIN A, LISCHINSKI D, WEISSY. Colorization using optimization[J]. ACM Transactions on Graphics (TOG), 2004, 23(3): 689-694.
    [4]
    NEUMANN L, NEUMANNA. Color style transfer techniques using hue, lightness and saturation histogram matching[C]// Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imaging. Switzerland: Eurographics Association Aire-la-Ville, 2005: 111-122.
    [5]
    XIAO X, MA L. Gradient-preserving color transfer[C]//Computer Graphics Forum. Hoboken, NJ: Blackwell Publishing Ltd, 2009, 28(7): 1879-1886.
    [6]
    PITIE F, KOKARAM A C, DAHYOT R. N-dimensional probability density function transfer and its application to color transfer[C]// Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, 2005, 2: 1434-1439.
    [7]
    ABADPOUR A, KASAEI S. An efficient PCA-based color transfer method [J]. Journal of Visual Communication and Image Representation, 2007, 18(1): 15-34.
    [8]
    PITI F, KOKARAM A C, DAHYOT R. Automated colour grading using colour distribution transfer[J]. Computer Vision and Image Understanding, 2007, 107(1): 123-137.
    [9]
    TAI Y W, JIA J, TANG C K. Local color transfer via probabilistic segmentation by expectation-maximization[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, 1: 747-754.
    [10]
    FRIGO O, SABATER N, DEMOULIN V, et al. Optimal transportation for example-guided color transfer[C]// Asian Conference on Computer Vision. Springer International Publishing, 2014: 655-670.
    [11]
    FERRADANS S, PAPADAKIS N, PEYR G, et al. Regularized discrete optimal transport[J]. SIAM Journal on Imaging Sciences, 2014, 7(3): 1853-1882.
    [12]
    YANG Y, ZHAO H, YOU L, ET AL. Semantic portrait color transfer with internet images[J]. Multimedia Tools and Applications, 2017, 76(1): 523-541.
    [13]
    SOLOMON J, DE GOES F, PEYR G, et al. Convolutional Wasserstein distances: Efficient optimal transportation on geometric domains[J]. ACM Transactions on Graphics (TOG), 2015, 34(4): No. 66.
    [14]
    VILLANI C. Topics in Optimal Transportation[M]. Providence, RI: American Mathematical Society, 2003.
    [15]
    KRIZHEVSKY A, SUTSKEVER I, HINTONG E. ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems 25. Lake Tahoe, Nevada: Neural Information Processing Systems Foundation, 2012: 1097-1105.
    [16]
    SERMANET P, EIGEN D, ZHANG X, et al. Overfeat: Integrated recognition, localization and detection using convolutional networks[DB/OL]. arXiv.org: arXiv:1312.6229, 2013.
    [17]
    LONG J, SHELHAMER E, DARRELLT. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 3431-3440.
    [18]
    HUANG G B, RAMESH M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments[R]. Amherst, MA: University of Massachusetts, Amherst, 2007: Technical Report 07-49.
    • Supplements(0)
    • Related Articles
    • Track Citations
  • 加载中

Catalog

    |
    Get Citation
    PDF
    XML
    [1]
    SHIRLEY P, ADHIKHMIN M, GOOCH B, et al. Color transfer between images[J]. IEEE Computer Graphics and Applications, 2001, 21: 34-41.
    [2]
    WELSH T, ASHIKHMIN M, MUELLER K. Transferring color to greyscale images[J]. ACM Transactions on Graphics (TOG), 2002, 21(3): 277-280.
    [3]
    LEVIN A, LISCHINSKI D, WEISSY. Colorization using optimization[J]. ACM Transactions on Graphics (TOG), 2004, 23(3): 689-694.
    [4]
    NEUMANN L, NEUMANNA. Color style transfer techniques using hue, lightness and saturation histogram matching[C]// Proceedings of the First Eurographics conference on Computational Aesthetics in Graphics, Visualization and Imaging. Switzerland: Eurographics Association Aire-la-Ville, 2005: 111-122.
    [5]
    XIAO X, MA L. Gradient-preserving color transfer[C]//Computer Graphics Forum. Hoboken, NJ: Blackwell Publishing Ltd, 2009, 28(7): 1879-1886.
    [6]
    PITIE F, KOKARAM A C, DAHYOT R. N-dimensional probability density function transfer and its application to color transfer[C]// Tenth IEEE International Conference on Computer Vision (ICCV’05) Volume 1. IEEE, 2005, 2: 1434-1439.
    [7]
    ABADPOUR A, KASAEI S. An efficient PCA-based color transfer method [J]. Journal of Visual Communication and Image Representation, 2007, 18(1): 15-34.
    [8]
    PITI F, KOKARAM A C, DAHYOT R. Automated colour grading using colour distribution transfer[J]. Computer Vision and Image Understanding, 2007, 107(1): 123-137.
    [9]
    TAI Y W, JIA J, TANG C K. Local color transfer via probabilistic segmentation by expectation-maximization[C]// 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, 1: 747-754.
    [10]
    FRIGO O, SABATER N, DEMOULIN V, et al. Optimal transportation for example-guided color transfer[C]// Asian Conference on Computer Vision. Springer International Publishing, 2014: 655-670.
    [11]
    FERRADANS S, PAPADAKIS N, PEYR G, et al. Regularized discrete optimal transport[J]. SIAM Journal on Imaging Sciences, 2014, 7(3): 1853-1882.
    [12]
    YANG Y, ZHAO H, YOU L, ET AL. Semantic portrait color transfer with internet images[J]. Multimedia Tools and Applications, 2017, 76(1): 523-541.
    [13]
    SOLOMON J, DE GOES F, PEYR G, et al. Convolutional Wasserstein distances: Efficient optimal transportation on geometric domains[J]. ACM Transactions on Graphics (TOG), 2015, 34(4): No. 66.
    [14]
    VILLANI C. Topics in Optimal Transportation[M]. Providence, RI: American Mathematical Society, 2003.
    [15]
    KRIZHEVSKY A, SUTSKEVER I, HINTONG E. ImageNet classification with deep convolutional neural networks[C]// Advances in Neural Information Processing Systems 25. Lake Tahoe, Nevada: Neural Information Processing Systems Foundation, 2012: 1097-1105.
    [16]
    SERMANET P, EIGEN D, ZHANG X, et al. Overfeat: Integrated recognition, localization and detection using convolutional networks[DB/OL]. arXiv.org: arXiv:1312.6229, 2013.
    [17]
    LONG J, SHELHAMER E, DARRELLT. Fully convolutional networks for semantic segmentation[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015: 3431-3440.
    [18]
    HUANG G B, RAMESH M, BERG T, et al. Labeled faces in the wild: A database for studying face recognition in unconstrained environments[R]. Amherst, MA: University of Massachusetts, Amherst, 2007: Technical Report 07-49.
    Recommended articles
    TrendMD
    Volume 48 Issue 2 page: 148-153
    Cover

    Article Metrics

    Article views (685) PDF downloads(261)
    Proportional views

    Copyright © Editorial Office of JUSTC, All Rights Reserved. 皖ICP备05002528号

    Supported by: Beijing Renhe Information Technology Co. Ltd    

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return