In order to detect fires in ancient buildings, a device for automatic smoke and fire detection and recognition based on the improved YOLO algorithm instead of human and traditional sensors is designed and analyzed around the clock. FPN is utilized to solve the problem of the existence of small flames that are not easy to be detected in ancient buildings, and a flame detection system is established on the basis of the CNN model. In order to improve the accuracy of detection and reduce the probability of false alarms, electronic sensors are inserted into this system in order to further detect signals such as temperature and smoke. The experimental results show that the improved algorithm model improves the accuracy and real-time performance of fire detection, and the recognition accuracy can reach more than 96%.