The multi-primary-color display faces the problem of color conversion between the device-dependent color space and the standard color space. A color conversion method based on BP-neural-network with luminance classification is proposed, and the conversion model from CIE standard (X,Y,Z) space to multi-primary-color (K1,K2,…,Kn) space is established. In this model, the (X,Y,Z) color space is decomposed into several two-dimensional subspaces according to the luminance factor Y of the training samples, and a series of BP-networks are established according to the luminance factors. Thus, this model overcomes the metamerism problem due to the color conversion from low dimensional space to high dimensional space. The validation experiment for this model is carried out using a five-primary-color LED display system. Firstly, on the bases of the actual chromaticity parameters of the five-primary-color LED display system, a linear conversion model for the color conversion between (K1,K2,K3,K4,K5) color space and (X,Y,Z) color space is established. Furthermore, the typical training set and testing set are generated according to the minimum color difference matching principle, and the BP neural networks are trained and tested. The results show that the average CIE1976L*a*b* color difference of the training set is below 6.37, and it needs further improvement. This study provides an effective approach for the color conversion of multi-primary-color display.