An optical system for panoramic surveillance cameras that can achieve clear all-weather imaging was designed. A panoramic annular structure was adopted, which was divided into two parts: camera head unit and relay lens unit. The camera head unit completed the full field of view target search, and the relay lens unit performed secondary imaging of the intermediate virtual image formed by the head unit and converged it on the detector. The design adopted multiple structural optimization methods to achieve visible light and near-infrared dual band imaging. The field of view of the optical system is 360°× (40°~100°), the focal length is ?2.75 mm, and the Fnumber is 3.28. The design results show that the modulation transfer function (MTF) value is close to the diffraction limit at the full field of view, the diffuse spot radius of each field of view is less than the selected CCD pixel size, the distortion is less than 2%, and the day-night defocus amount is less than 0.002 mm, which can meet the requirements of all-weather and panoramic surveillance.