In the 3D reconstruction of color structured light, the non-linear coupling of systems, the topological structure of the test surface, and other factors will affect the decoding of structured light, which can lead to the missed detection of modulation fringes and the false recognition of color codes. To solve the problem, we proposed the 3D reconstruction method of color-coded structured light. First, filtering differential projection was performed on the modulation fringes through the YUV color channel, and then the center feature lines of the fringes were extracted after an analysis of the waveform distribution of the fringes. Furthermore, the information values of color codes of the fringes were accurately acquired by virtue of the color clustering method. Finally, in order to establish the corresponding relationship between the codewords of coded fringes and those of modulation fringes, we put forward an optimal matching method based on the combination of sequence features, and combined with the mathematical model of binocular depth perception, we solved the three-dimensional information value of the codewords of coded fringes. The experimental results show that this method has a low rate of missed detection, a high recognition rate of color codes, and strong anti-interference and robustness.