A compact multi-spectral imaging system was designed by using the model of aperture segmentation in order to solve the problem of volume and complex optical path, and this system consists of the front optical element, array aperture, micro lens array, filter and image sensor, with the overall size less than the12.4×8×8(mm3). Some problems, including large vertical axis chromatic aberration, small field of view and, low utilization of light energy, were all overcame by using the array aperture. The system has a field of view of 20°, F/#0.4, focal length of 2.5mm and total length of 12.4mm in the wavelength range of 480～650nm. The single-channel MTF is greater than 0.35 at 153lp/mm, and the distortion is less than 0.23%. Compared with the existing technology, the system takes into account the pixel, spectral and time resolution, achieving a pixel resolution of 500×400 and a spectral resolution of 6.8nm for a single channel. Compared with the computational reconstruction imaging system, the direct imaging ensures the high temporal resolution of the system, and shows the characteristics of miniaturization, light weight and low cost.