Abstract: Portfolio theory has been extensively studied and applied in finance. To determine the optimal portfolio weight under the global minimum variance strategy, it is necessary to estimate both the covariance matrix and its inverse. However, the high dimensionality and heavy-tailed nature of financial data pose significant challenges to this estimation. In this study, we propose a method to estimate the Gini covariance matrix by introducing a low-rank and sparse correlation structure, as an alternative to the traditional sample covariance matrix. Our approach employs a factor model to capture the low-rank structure, combined with thresholding rules to achieve the final estimation. We demonstrate the consistency of our estimators and validate our approach through simulation experiments and empirical portfolio analyses. Simulation results show that our method is highly applicable across a variety of distributional scenarios. Furthermore, empirical portfolio analysis indicates that our method can construct portfolios with superior performance.