For detecting mirror plane defects, fringe modulation is generally regarded as an effective method. To improve the accuracy of defect detection, this paper proposes a sector fringe-based modulation testing method. The proposed method is more compatible with the defect direction and suitable for many types of defects. A new quality map is used based on deformed fringe modulation and phase gradient deviation to identify the locations of defects. Eight groups of experiments are conducted and compared.Results show that the modulation and phase gradient quality map can suppress the selectivity of defect directions and the testing accuracy of defects can reach a minimum size of 0.07 mm. Compared with the traditional machine vision method and horizontal/vertical/circular fringe-based methods, the proposed sector fringe modulation method offers state-of-the-art sensitivity and higher accuracy.