The existing multi-band imaging system has the problems of large volume, high power consumption, and difficulty in integrating design. To address these challenges, we proposed a three-band co-aperture imaging optical system based on single sensor. First, a 1×2 multi-band lens array in the aperture stop of the optical system is designed. This array effectively captures both the visible and short-wave infrared bands simultaneously in a single image plane. In addition, the imaging position deviation of the center wavelength of both bands are controlled within one pixel, resulting in dual-band fusion imaging. To address the issue of different diffraction limits in multi-band imaging, we propose to use the joint optimization method to simultaneously control the off-axis offset and aperture size of the split channel lens array. On the above basis, we suggest utilizing a dual electric diaphragm to control the switching speed of the three imaging channels. Finally, a three-band co-aperture optical system based on single sensor with a focal length of 30 mm and operating bands ranging from 480 to 900 nm, from 900 to 1700 nm, and from 480 to 1700 nm is designed. The system exhibits multiple advantages, such as excellent imaging quality, a compact structure, no moving optical elements, and a rapid switching speed of the imaging band, as indicated by the design and analysis results.