The unbalanced defocus blur of the left and right images leads to stereo matching failure in a binocular stereo vision system. In order to train a neural network that can deal with the image blur, this paper constructs an unbalanced defocus stereo vision dataset by adding the blur varying with the depth using a normalized blur level (NBL) based layered depth-of-field rendering algorithm and taking the FlyingThings-Stereo image pair dataset as an example. The proposed dataset can provide the unbalanced defocus stereo images and be used to train deblurring or stereo matching networks. When training the deblurring network, the dataset provides blurry and clear stereo images to the network's input and output ends. When training stereo matching network, fuzzy stereo pairs and parallax truth values are provided to the input and output ends of the network. The network is verified by synthetic and real-scene data after it is trained. Results show that the proposed dataset can effectively train the deblurring and stereo matching neural networks and enables their ability to cope with defocus blur, so as to achieve the image deblurring and stereo matching based on blurry images.