The conventional optical storage system uses Run Length Limited (RLL) modulation technology, which is limited by the optical diffraction limit and has great difficulty in capacity enhancement. Multi-order optical storage technology uses a new multi-order recording medium or multi-order modulation technology, which can significantly increase the density and capacity of optical storage without changing the optical parameters. In order to improve the density and capacity of high-order optical storage, the multi-order modulation coding optimization method is studied for optical disc storage, and the 4th-order RLL (1,7) modulation coding method is proposed, which can improve the maximum recording density up to 2.4bits, increase the capacity of optical disc by 50%, and have certain error correction capability compared with the 17PP coding method currently used in Blu-ray disc. The maximum recording density can be 2.4bits, the disc capacity can be increased by 50%, and the error correction capability can be improved.