Ammonia gas is an important basic industrial raw material, and realizing its non-contact detection is of great significance for the timely detection of ammonia gas leaks to avoid major safety incidents. Aiming at the shortcoming of conventional ammonia leak detection devices that can only respond when ammonia diffuses to a certain range and makes contact with a sensor, a Shuffling Self-Attention Network (SSANet) model is proposed to realize the infrared non-contact detection of ammonia leaks. Due to the high noise and low contrast of ammonia leakage images obtained by infrared cameras, an infrared detection dataset of ammonia leakage was established through non-local mean denoising and contrast-limited adaptive histogram equalization preprocessing. On the basis of YOLOv5s, the SSANet model uses the K-means algorithm to cluster and analyze the candidate frame suitable for the infrared detection of ammonia gas leakage to preset the model’s parameters. Using the lightweight ShuffleNetv2 network, the depth of 3×3 in the Shuffle Block can be adjusted. The separate convolution kernel is replaced with a 5×5 depth, and the feature extraction network is reconstructed with an SK5 Block containing a new convolution module, which makes the model size, calculation and parameters non-intensive while improving the detection accuracy. The Transformer module is used instead of its original version. The C3 module in the network bottleneck module is replaced by Transformer module to realize the bottom-up fusion of multi-head attention in the leake area, and further improves the detection accuracy. The experimental results show that the size and parameter requirements of the SSANet model are reduced by 76.40% and 78.30%, respectively, to 3.40 M and 1.53 M compared with the basic model of YOLOv5s; the average detection speed of a single image is increased by 1.10% to 3.20 ms; and the average detection accuracy is increased by 3.50% , reaching 96.30%. We provide an effective detection algorithm for the development of a non-contact detection device for ammonia leakage to ensure the safe production and stable operation of ammonia-related enterprises.