When the phase-shifting interferometer is disturbed by large vibration, it will not be able to accurately obtain the better interference group image. In order to solve this problem, a vibration detection method is proposed based on interference fringe image analysis.The proposed method is divided into two parts. From the perspective of image processing, the characteristics of the fringe shape, number, and phaseshift direction in the interferogram are analyzed, and the relationship between the characteristics and the low frequency, high amplitude vibration is obtained. Based on this, we can judge whether the interferogram is affected by low frequency vibration. From the perspective of statistics, the relationship between the phaseshift characteristics of the interferogram group and the mid-high frequency, low amplitude vibration is analyzed. By calculating the phase shift stability and accuracy of the interference group picture and evaluating the overall phase shift quality, we can judge whether the interferogram group is affected by mid-high frequency vibration. Based on the above method, the corresponding algorithm has been developed. After repeated experiments and verification, the method can intelligently judge and screen out the interference group pictures that are less disturbed by vibration when obtaining the sequence interference group pictures. Then we can calculate accurate wavefront information based on the obtained high-quality interferogram group. This method improves the anti-vibration ability of the system and is of significance to the practical application of phase-shifting interference technology.