To resolve the problem of detection of similar foreign materials in the complex background, a detection method based on transmission/reflection spectral imaging was proposed. According to the method, the different imaging model was constructed according to the different transmission/reflection characteristic discrimination between foreign materials and background. In different imaging model, the optimal illumination wavelength that distinguished the foreign materials from background was selected, and a transmission/reflection spectral imaging system was established. In actual application of detection of foreign materials in cotton, the image features of similar foreign materials from those of background were acquired by using optimal wavelength transmission/reflection spectral imaging system, and then the image targets were extracted by using adaptive thresholding image processing. The experiment result indicates that the detection ration is 91% under the condition of reflection illumination coefficient 0.74 and transmission illumination coefficient 0.83, and the method is a novel practical mean for detection of similar foreign materials currently.