Owing to the necessity and importance of the three-dimensional (3-D) displacement measurement technique in the field of engineering, a system based on a single CCD camera and a method of digital image correlation is developed. With regard to the relationship between the center and the slope of the displacement vector and displacements in and out-of the object plane, a least-squares fit method is employed to calculate constant terms (corresponding to the in-plane component) and the first order terms (corresponding to the out-of-plane component), and the three-dimensional displacement components are separated. On the theoretical basis of the pin-hole camera imaging model, a numerical simulation and an experiment on three-dimenaional translation of a silicon wafer are conducted. The simulation method of speckle images associated with the three-dimensional rigid body displacement is developed, which is further employed to validate the effectiveness and precision of the iterative correlation method based on affine transformation. Simulated and experimental results demonstrate that both in-plane and out-of-plane displacements can be accurately retrieved with the proposed method, and the maximum error is 5%.