InGaAs NIR detectors are widely used in aerospace, military and civilian fields. In order to realize the intelligence of InGaAs detectors, combined with face detection applications, an ultra-lightweight InGaAs NIR face detection algorithm that can be deployed in low-power mobile smart devices is proposed. This paper mainly studies the problems of few NIR face samples and low-power device deployment, and uses transfer learning and binary quantization to train the network. The algorithm first realizes a pre-trained face detection network based on SSD through a large-scale visible light face dataset. Then, the binary quantization scheme is used to greatly compress the network parameter space size and calculation amount, but the network accuracy is reduced at the same time. In order to further improve the effect of network binary quantization, this paper introduces feature mean information for the binary quantization process and makes up for the loss of accuracy in the form of adversarial convolution. Finally, the algorithm fine-tunes the pre-trained binary network through small-scale NIR face data to achieve the final network. The binarization face detection network implemented in this paper can achieve an average accuracy of 71.18% in the collected NIR face verification set.