Fig. 1. Pinhole imaging to optoelectronic digital imaging

Fig. 2. Schematic diagram of spatio-temporal coding-based modulation at different positions in imaging chain

Fig. 3. Sampling limit and optical diffraction limit of imaging system

Fig. 4. Relationship between resolution and sensitivity based on physical mechanism of photoelectric conversion. (a) Detectors with large pixel sizes can capture more photons within the same integration time, generating images with high signal-to-noise ratios but low resolution; (b) detectors with small pixel sizes can capture fewer photons within the same integration time, generating images with low signal-to-noise ratios but high resolution

Fig. 5. Pixel super-resolution imaging technology-based classification

Fig. 6. Principle diagram of spatio-temporal encoding modulation in different imaging links. (a) Aperture plane-based amplitude encoding modulation; (b) aperture plane-based phase encoding modulation; (c) aperture plane-based metasurface encoding modulation; (d) focal plane-based micro-scanning encoding modulation; (e) focal plane-based optical element encoding modulation; (f) focal plane-based camera array encoding modulation; (g) focal plane-based programmable device encoding modulation; (h) focal plane-based fixed mask encoding modulation; (i) focal plane-based heterogeneous pixel detector encoding modulation

Fig. 7. Classification of aperture plane-based spatio-temporal encoding methods and their representative works. (a) Super-resolution imaging via amplitude encoding modulation, proposed by Slinger et al.^[18-19]; (b) super-resolution imaging via phase encoding modulation, proposed by Yi et al.^[20]; (c) super-resolution imaging via metalens modulation, proposed by Li et al.^[21]; (d) super-resolution imaging via metalens modulation, proposed by Zhang et al.^[22]

Fig. 8. Related work on pixel super-resolution imaging technology achieving aperture plane coding control using amplitude masks. (a)‒(b) Mid-wave infrared super-resolution imaging system based on micro-opto-electro-mechanical systems proposed by Gorden et al.^[23] and its comparison of results before and after super-resolution; (c)‒(d) mid-wave infrared super-resolution imaging system based on fixed coded aperture imaging proposed by Slinger et al.^[18-19] and its comparison of results before and after super-resolution

Fig. 9. Related work on pixel super-resolution imaging technology achieving aperture plane coding control using phase masks. (a)‒(b) Wavefront coding-based super-resolution imaging system proposed by Zhao et al.^[24] and its comparison of results before and after super-resolution; (c)‒(d) experimental system for rotating multi-frame pixel super-resolution imaging based on phase coding proposed by Yi et al.^[20]and its comparison of results before and after super-resolution

Fig. 10. Related work on pixel super-resolution imaging technology achieving aperture plane control using metasurface devices. (a)‒(b) Single-pixel super-resolution imaging system based on a reconfigurable metasurface proposed by Li et al.^[21] and its comparison of results before and after super-resolution; (c)‒(d) remote Fourier ptychographic super-resolution imaging system based on a conjugate quadratic phase metasurface pair proposed by Zhang et al.^[22] and its comparison of results before and after super-resolution

Fig. 11. Classification of focal plane-based spatio-temporal encoding methods and their representative works. (a) Super-resolution imaging via micro-scanning, proposed by Ben-Ezra et al.^[30]; (b) super-resolution imaging via programmable aperture modulation, proposed by Chen et al.^[32]; (c) super-resolution imaging via fixed mask modulation, proposed by Xiao et al.^[33]; (d) super-resolution imaging via heterogeneous pixel detectors, proposed by Shi et al.^[34]

Fig. 12. Related work on pixel super-resolution imaging technology achieved through active micro-scanning focal plane control using micro-scanning devices. (a)‒(b) “Jitter camera” system prototype proposed by Ben-Ezra et al.^[30] and its comparison of results before and after super-resolution; (c)‒(d) experimental diagram of infrared micro-scanning optical system proposed by Zhang et al.^[31] and its comparison of results before and after super-resolution

Fig. 13. Related work on pixel super-resolution imaging technology achieved through active micro-scanning focal plane control using optical elements. (a)‒(b) Super-pixel scanning experimental system proposed by Sun et al.^[37] and its comparison of results before and after super-resolution; (c)‒(d) Risley prism-based scanning system proposed by Gui et al.^[38] and its comparison of results before and after super-resolution

Fig. 14. Related work on pixel super-resolution imaging technology achieving focal plane control using camera arrays. (a)‒(b) Large-scale compound-eye array imaging system proposed by Bennett Wilburn et al.^[44] and its comparison of results before and after super-resolution; (c)‒(d) four-camera array system proposed by Yang et al.^[45] and its comparison of results before and after super-resolution

Fig. 15. Related work on pixel super-resolution imaging technology achieving focal plane control using programmable devices. (a)‒(b) Mid-wave infrared compressive sensing imaging system proposed by Mahalanobis et al.^[48] and its comparison of results before and after super-resolution; (c)‒(d) system based on focal plane array compressive sensing proposed by Chen et al.^[32] and its comparison of results before and after super-resolution

Fig. 16. Related work on pixel super-resolution imaging technology achieving focal plane control using fixed masks. (a)‒(b) Principle of compressive sensing infrared imaging system based on a focal plane coded aperture mask proposed by Xiao et al.^[33] and its comparison of results before and after super-resolution; (c)‒(d) focal plane coding system based on a translating coded mask proposed by Mahalanobis et al.^[50] and its comparison of results before and after super-resolution

Fig. 17. Related work on pixel super-resolution imaging technology achieving focal plane control using heterogeneous pixel detectors. (a)‒(b) Schematic diagram of a heterogeneous pixel detector based on Penrose tiling proposed by Ben-Ezra et al.^[51] and its comparison of results before and after super-resolution; (c)‒(d) schematic diagram of a heterogeneous pixel detector satisfying octic group transformations proposed by Shi et al.^[34] and its comparison of results before and after super-resolution

Fig. 18. Development history of data-driven deep learning pixel super-resolution imaging technology. (a) SRCNN structure proposed by Dong et al.^[52-53]; (b) FSRCNN structure proposed by Dong et al.^[54]; (c) VDSR network structure proposed by Kim et al.^[55]; (d) EDSR network structure proposed by Lim et al.^[57]; (e) SRGAN structure proposed by Ledig et al.^[65]; (f) ZSSR network structure proposed by Shocher et al.^[66]; (g) SwinIR network structure proposed by Liang et al.^[68]; (h) gated fusion Transformer network structure proposed by Bian et al.^[69]