With the restriction of detector resolution in EDXRF, the peaks will be widened, and the overlapping of several peaks cannot be neglected especially in multi elements sample. The process of achieving the intensity of pure peaks, named spectrum unfolding is difficult. There are three unfolding methods traditionally, ROI integration, pure element peeling, and the coefficient method. Both of the methods cannot unfold the complex EDXRF spectrum perfect. In this work, we introduce an optimized algorithm to EDXRF spectrum unfolding process. Pure peaks, processed by applying the smooth and debackground operation to raw spectrum, can be considered as the sum of several Gaussian peaks, which can be formulated by Gaussian functions with 3 parameters each, height, width and position. By alternating the values of the 3N parameters within certain boundary conditions, the error between theory function and raw spectrum can be smaller and smaller. In mathematics, this alternating process can be finished by Optimize algorithm. When we developed the algorithm by 3N parameters Conjugate Gradient Method and coding the program, an EDXRF spectrum of Pr/Nd solution was studied. Sum of 11 Gaussian peaks with 33 parameters was very well closed to the raw spectrum composed of 11 L-series peaks of Pr and Nd, and the error calculated changed from 37.645 to 1.699 4. The innovative of this paper is the application of optimized method in complex EDXRF spectrum unfolding process.