Fig. 1. Technological transformations in optical perception

Fig. 2. Basic architecture of optical perception

Fig. 3. Workflow of optical encoding technology

Fig. 4. Coded optical imaging for functional expansion. (a) Rainbow 3D camera^[15]; (b) scanning spectral imaging approaches^[18]; (c) snapshot colored compressive spectral imager^[19]; (d) single-dispersive-element coded aperture snapshot spectral imaging system^[20]; (e) programmable pixel compressive camera^[21]; (f) snapshot spatial-temporal compressive imaging system incorporating polarization information^[22]

Fig. 5. Coded optical imaging for performance enhancement. (a) Single-pixel terahertz imaging system^[25]; (b) principle and microscope of structured illumination microscopy^[26-27]; (c) Fourier ptychographic imaging system^[28]; (d) action recognition pipeline based on optical pixel-wise encoding^[29]

Fig. 6. Optical pre-sensing computing creates new architectures for intelligent perception

Fig. 7. Mathematical model of a neuron^[34]

Fig. 8. Linear computing in diffractive optical neural networks based on phase masks or SLMs. (a) All-optical diffractive deep neural networks^[36]; (b) reconfigurable diffractive processing unit^[37]

Fig. 9. Linear computing in diffractive optical neural networks based on metasurfaces. (a) Multifunctional metasurface-based diffractive neural networks^[38]; (b) programmable diffractive deep neural network based on a digital-coding metasurface array^[39]; (c) metasurface diffractive optical neural network for simulating human-level decision-making and control^[40]

Fig. 10. Linear computing realized via 4f optical system. (a) Convolution computing based on phase modulation in 4f system^[41];(b) convolution computing based on amplitude modulation in 4f system^[42]; (c) optical pooling based on 4f system^[43]

Fig. 11. Simplified machine vision based on incoherent light amplitude modulation. (a) Lensless opto-electronic neural network^[44]; (b) face recognition system with a mask-encoded microlens array^[46]

Fig. 12. Multilayer pre-sensing computing based on nonlinear activation. (a) Multilayer fully connected computational architecture based on nonlinear activation^[51]; (b) multilayer convolutional computational architecture based on nonlinear activation^[52]

Fig. 13. Training methods for hardware parameters in optical neural networks^[54]

Fig. 14. Training architecture for optical neural networks based on forward-forward algorithm^[54]

Fig. 15. Realization of optical nonlinearity and optical compression via multiple scattering^[59]

Fig. 16. Intelligent recognition systems based on phase modulation in incoherent light scenarios. (a) Privacy-preserving facial depression recognition technology^[60]; (b) privacy-preserving scene description system^[61]

Fig. 17. Metasurface folded lens system for ultrathin cameras^[62]