Aiming at the shortcomings of current image feature point face recognition algorithms such as low matching accuracy, a face recognition algorithm based on convolutional neural network is proposed. The algorithm uses traditional algorithm operators to fuse convolutional neural networks for recognition. First, the idea of local receptive field is used to segment the overall image to obtain a local image set. Each local image pixel in the set is stored in the pixel matrix Ai. Then the convolution operation is performed on each local image to obtain the intrinsic feature relationship between the local images, and it is stored in the Bi matrix and pooled for feature mapping. Finally, the network weighting coefficients are trained and the recognition results are obtained. Experimental results show that compared with other algorithms, the proposed algorithm improves the problem of low matching accuracy of image feature points of the original algorithm, and verifies the effectiveness of the proposed algorithm.