Traditional single speckle pattern matching algorithms always suffer from the low measurement accuracy and cannot be used to measure complex surface objects. A speckle projection profilometry with deep learning was proposed to realize the pixel-by-pixel matching. The siamese convolutional neural network structure was applied and extended where the main speckle pattern and the auxiliary speckle pattern were fed into the neural network patch by patch. It was expected that the feature from the speckle pattern patches could be extracted by the convolution operation. In this way, the features were fused and the matching coefficient between the two patches was obtained, which could be further used to formulate the disparity data and then the three-dimensional (3D) object was reconstructed. The experiment results demonstrate that with the proposed method 3D measurement with an accuracy of about 290 μm could be achieved through a single speckle pattern.