Traditional genetic algorithms (GA) are of good ability of global search because of its parallelism, while tabu search algorithm is of fast convergence and great local search capacity. Combining the advantages of genetic algorithms and tabu search algorithm, a new algorithmgenetic/tabu hybrid optimization algorithm is proposed to design multilayer coatings. The theory of tabu search is imported into the crossover and mutation processes of GA, which makes the algorithm have the ability of memorizing. Better designs are obtained using genetic/tabu hybrid optimization algorithm to design antireflection coatings and beam splitter than the traditional GA. The results indicate the effectiveness and reliability of the algorithm.