The phase retrieval algorithms can be used to recover the field at the distal end of a fiber from the intensity at the proximal end of the fiber. The response of the fiber can be described by the transmission matrix. In the experiment, a sufficient number of samples are sampled from the output intensity distribution with different input conditions to measure the transmission matrix. Obviously, the position distribution of sampling points, including the sampling number and interval, affects the measurement of the transmission matrix, and the accuracy and efficiency of the phase retrieval algorithm are related to the transmission matrix. We propose that the sampling interval should be greater than the speckle size to ensure the independence of different rows of the transmission matrix; therefore, image quality can be guaranteed with fewer sampling points at higher reconstruction efficiency. The experimental results show that when the sampling interval is less than the speckle size, the number of sampling points required for light field reconstruction decreases significantly with increasing sampling interval under the same image reconstruction quality. When the sampling interval is greater than the speckle size, the number of sampling points required changes slowly and finally remains approximately 3.5 times the number of pixels of the input image. When the sampling interval is fixed, with the increase in sampling points, the time consumed by the phase retrieval algorithm first decreases and then increases, so there is an optimal sampling interval and sampling points.