Diffuse Optical Tomography (DOT) is a new functional optical imaging technique that demonstrates the potential applications in breast tumor imaging and functional brain imaging. It has the advantages of being non-invasive, non-ionizing radiation, and providing physiological information about biological tissues. The image reconstruction process of DOT, that is, the inverse problem, is ill-posed, resulting in a low degree of quantification and spatial resolution of the reconstructed image. Traditional DOT image reconstruction methods cannot completely solve the problem of low imaging accuracy, mainly reconstruct the target with a regular shape (circle). In recent years, artificial neural networks have been widely used in the field of image reconstruction with their strong feature extraction and recognition capabilities. In this paper, Stacked Auto-Encoder (SAE) neural network is proposed to improve the DOT image quality. SAE is a relatively simple model where fewer network parameters need to be adjusted, and therefore the training speed is fast. SAE network consists of two autoencoders, which are unsupervised learning neural network models, including encoder and decoder. The encoder works by extracting the features of the input data to the hidden layer, while the decoder reconstructs the input data from the hidden layer. A fully connected layer after the autoencoder is added as the output layer of SAE. The SAE network training process consists of two stages. The first stage is unsupervised pre-training for getting initial weights and bias values. The second stage is that based on the principle of error backpropagation, the network minimizes the loss function by calculating the mean squared error between the predicted output value and the expected output value, then optimizes the weights and biases in the network model, and finally reconstructs the optical parameter image. For practical applications, anatomical prior information incorporated into SAE neural network is utilized to reconstruct DOT images with the targets of different shapes (circle and ellipse). To simulate the tumor in breast tissues, a two-dimensional circular phantom with a radius of 40 mm is used to simulate background tissue, and the circular and elliptical targets are respectively embedded in the background phantom to simulate breast tumors. Sixteen coaxial source-detector optodes are equally arranged around the phantom boundary. For each illumination, except the 4 detectors nearest to the source position, the remaining 11 detectors are used to measure the light intensities on the boundary, leading to a total of 16×11 (176) measurements. To more closely simulate the real case, 40 dB noise is added to the measurement data. Based on the numerical optical phantoms, the corresponding MRI images with the size of 51×51 are simulated to provide anatomical prior information. In SAE network, the normalized measurement data and the normalized gray values of the MRI image are utilized as the neural network inputs, and the optical parameters of the finite element nodes are used as the outputs to obtain DOT image. In order to verify the feasibility and effectiveness of the proposed method, a series of numerical simulation experiments with and without prior information are carried out. The experimental results are assessed and compared quantitatively using the Mean Absolute Error (MAE), Mean Square Error (MSE) and Quantitativeness Ratio (QR). The experimental results of the single circular target with different absorption contrast and different size show that the reconstruction results of the prior information-based SAE method are closer to the real image, and when the absorption contrast is low (1.5), the MAE of the fusion prior information method is reduced by 62%, the MSE is reduced by 11%, and the QR value is reduced from 139% to 107% which is closer to 100%, compared with no prior information method. It is worth mentioning that when the absorption contrast is larger than 1.5 and the radius is larger, both methods can achieve better image reconstruction quality since it is easier to recover the target in theory. For the image reconstruction of single and double targets of elliptic shapes, the prior information-based SAE method can accurately recover the size and position of the target and demonstrates high noise robustness and quantitativeness. Especially, when the absorption contrast is relatively small, the prior information-based SAE method can reduce the prediction error effectively. In addition, the quantitative analysis and comparison show that the MAE and MSE are significantly reduced by using the prior information-based SAE method. We find that the MAE is reduced by at least 8%, MSE is reduced by at least 5%, and the value of QR is closer to 100%. The comprehensive evaluation indicates that our proposed method can effectively improve the imaging accuracy and the quality of DOT reconstruction images.