Recently optical coherence tomography (OCT) has obtained a fast development and application in clinic due to its "optical biopsy" properties. In addition, OCT could also provide molecular information after using contrast agents. However, single modal OCT technique has its disadvantages, such as poor penetration depth. These can be avoided by multi-modal imaging technique, which will also help to make full use of OCT system and to get a full understanding of organism from different aspect. In this review, we summarized the development of multi-modal contrast agents for OCT, fluorescence imaging, photoacoustic imaging and magnetic resonance imaging, and discussed the future of multi-modal contrast agents.