Fig. 1. The proposed edge acceleration framework for in situ reconstruction of computational imaging. Operator reconfiguration is first conducted for the selected AI chip. Then, the sensitivity of pruning and quantization is characterized for each layer or block. Next, sensitivity-aware pruning and quantization are sequentially performed following the guidance. Finally, the compressed model is deployed to the chip for acceleration.

Fig. 2. (a) The prototype of used single-lens infrared computational camera. (b) The architecture of the original network used in this work.

Fig. 3. Edge inference time of operators optimized by default.

Fig. 4. (a)–(f) Performance degradation caused by pruning at different stages. The black lines indicate the performance of the unpruned model without pruning, while the colored lines represent the performance after pruning the corresponding proportion of each block individually. For example, the red marker on the horizontal axis of D1 denotes the performance after pruning 25% of the D1 block, based on the original model.

Fig. 5. (a), (b) Performance degradation caused by quantization for different blocks. The black lines indicate the performance of the unpruned model with FP16 quantization, while the colored lines represent the performance of both the unpruned model and the uniformly 50%-pruned model under INT8 quantization, applied to each block individually. For example, the green marker on the horizontal axis of D1 denotes the performance after applying INT8 quantization to the D1 block based on the original model.

Fig. 6. (a) The edge pruning sensitivity results. (b) The edge quantization sensitivity results.

Fig. 7. Ablation experimental results on reconstruction focus on the details of clouds in local areas. Sensitivity-aware pruning restores finer texture details within clouds compared to uniform pruning, closely matching the performance of the unpruned network.

Fig. 8. Ablation experimental results on reconstruction focus on the details of vegetation in local areas. Sensitivity-aware pruning restores finer texture details of branches compared to uniform pruning, closely matching the performance of the unpruned network.

Fig. 9. Experimental results of MTF testing. The MTFs across various fields at the Nyquist frequency (42 lp/mm) all exceed 0.5, showing excellent high-frequency performance.

Fig. 10. Outdoor experimental assessment with real-time on-chip reconstruction.

Fig. 11. Experimental results of small infrared target tracking.