Existing restoration methods perform virtual restoration of computer-aided cultural relics with low accuracy and speed. To address this issue, a new reassembly method of cultural relics based on feature Point matching of fracture surface is proposed. First, the improved internal shape signature method is used to extract potential feature points of fragment fracture surfaces. Then, the covariance matrix of geometric features of adjacent feature points is calculated to construct feature descriptors. The logarithmic Euclidean Riemann method is then used as the similarity measure criterion, and the initial point pair set is obtained based on the bidirectional nearest neighbor method. The optimal matching set is obtained by eliminating mismatching pairs based on the canonical correlation analysis method. Finally, the least square method is used to calculate the rigid body transformation matrix to align the fragments and the iterative closest point algorithm is used to achieve precise alignment, thereby realizing fragment reassembly. Experimental results show that the proposed algorithm has fewer feature points compared with traditional algorithms; the descriptor is simple and robust, which effectively improves the efficiency and accuracy of fragment reassembly.