As a key component of laser scanning display system, it is an inevitable requirement for RGB combiner to be of miniaturization and high transmission efficiency applied in augmented reality. Waveguides formed by three different materials with high transparency in visible light band, such as silicon nitride, gallium nitride and SU8, were comparatively studied in aspects of refractive index difference, dispersion curve and single mode condition, as well as dimension, transmission efficiency and optical field distribution of multi-mode interference RGB waveguide combiners based on the waveguides. The results show that the performances of silicon nitride devices are somewhere between those of gallium nitride devices and SU8 devices. Gallium nitride devices have the largest core-cladding refractive index difference, the smallest single-mode cut-off size, and the shortest device length (2 000 μm). SU8 devices have the smallest core-cladding refractive index difference, the largest single-mode cutoff size, but the larger device length (3 600 μm) than that of gallium nitride devices. In addition, the average RGB transmission rate of the silicon nitride, gallium nitride and SU8 devices are 78%, 55% and 91%, respectively. The SU8 device dominates obviously. The optimized ultra-compact gallium nitride multi-mode interference RGB waveguide combiner is expected to be integrated to laser scanning monolithic systems, while SU8 multi-mode interference RGB waveguide combiner has great practical value in deformable systems due to its well flexibility. Those devices provide a solid technical foundation for the development of augmented reality systems towards microminiaturization and high transmission efficiency.