Laser & Optoelectronics Progress, Volume. 61, Issue 2, 0211030(2024)
Review of Optical Pre-Sensor Computing Technology and Its Satellite Remote Sensing Applications (Invited)
[1] Huang Z H, Fang H, Li Q et al. Optical remote sensing image enhancement with weak structure preservation via spatially adaptive gamma correction[J]. Infrared Physics & Technology, 94, 38-47(2018).
[2] Ye B, Tian S F, Ge J et al. Assessment of WorldView-3 data for lithological mapping[J]. Remote Sensing, 9, 1132(2017).
[3] Pan T. Technical characteristics of Gaofen-2 satellite[J]. Aerospace China, 3-9(2015).
[4] Li W, Liu X, Zhang W C et al. The application of deep learning in space-based intelligent optical remote sensing[J]. Spacecraft Recovery & Remote Sensing, 41, 56-65(2020).
[5] Shastri B J, Tait A N, de Lima T F et al. Photonics for artificial intelligence and neuromorphic computing[J]. Nature Photonics, 15, 102-114(2021).
[6] Fu T Z, Sun R, Huang Y Y et al. Review of on-chip integrated optical neural networks[J]. Chinese Journal of Lasers, 51, 0119001(2024).
[7] Lin X, Rivenson Y, Yardimci N T et al. All-optical machine learning using diffractive deep neural networks[J]. Science, 361, 1004-1008(2018).
[8] Chen B, Zhang Z Y, Dai T G et al. Photonic neural networks and its applications[J]. Laser & Optoelectronics Progress, 60, 0600001(2023).
[9] Hossain M S, Shahriar G M, Mahbubul Syeed M M et al. Region of interest (ROI) selection using vision transformer for automatic analysis using whole slide images[J]. Scientific Reports, 13, 11314(2023).
[10] Zhou Y, Li R Z, Yu X H et al. Progress in study and application of optical field modulation technology based on liquid crystal spatial light modulators (invited)[J]. Acta Photonica Sinica, 50, 1123001(2021).
[11] Dudley D, Duncan W M, Slaughter J. Emerging digital micromirror device (DMD) applications[J]. Proceedings of SPIE, 4985, 14-25(2003).
[12] Wang Y J, Chen H Z, Liu Y Y et al. Application of DMD to high dynamic scene image detection system[J]. Optics and Precision Engineering, 22, 2508-2517(2014).
[13] Luo H. Research on pixel-level dimming technology based on digital micromirror device[D](2020).
[14] Yang C. Optimization of DMD lithography imaging quality based on intelligent algorithm[D](2023).
[15] Sun Y Q, Hu Y, Wang Y Q et al. Analysis on aberration of digital micromirror device in convergent imaging path[J]. Acta Optica Sinica, 39, 0311001(2019).
[16] Li Y L, Fu X L[M]. Spatial light modulator and its application(1996).
[17] Chen H. Research on multimode fiber imaging based on phase modulation and deep learning[D](2022).
[18] Zeng Z. Research on key technology of optical surface shape measurement based on spatial light modulator[D](2023).
[19] Poleshchuk A G. Fabrication and application of diffractive optical elements[J]. Proceedings of SPIE, 7544, 75443L(2010).
[20] Danziger Y, Hasman E, Friesem A A et al. Multilevel diffractive elements for generalized wavefront shaping[J]. Optical Engineering, 35, 2556-2565(1996).
[21] Hu H Q, Zhou H, Xu Z H et al. Practical snapshot hyperspectral imaging with DOE[J]. Optics and Lasers in Engineering, 156, 107098(2022).
[22] Ryf R, Stuart H R, Giles C R. MEMS tip/tilt and piston mirror arrays as diffractive optical elements[J]. Proceedings of SPIE, 5894, 58940C(2005).
[23] Yang L L, Cui Q F, Liu T et al. Effects of manufacturing errors on diffraction efficiency for multilayer diffractive optical elements[J]. Applied Optics, 50, 6128-6133(2011).
[24] Kwon H, Arbabi E, Kamali S M et al. Single-shot quantitative phase gradient microscopy using a system of multifunctional metasurfaces[J]. Nature Photonics, 14, 109-114(2020).
[25] Xiong B, Liu Y, Xu Y H et al. Breaking the limitation of polarization multiplexing in optical metasurfaces with engineered noise[J]. Science, 379, 294-299(2023).
[26] Khorasaninejad M, Chen W T, Devlin R et al. Metalenses at visible wavelengths: diffraction-limited focusing and subwavelength resolution imaging[J]. Science, 352, 1190-1194(2016).
[27] Levanon N, Indukuri S R K C, Frydendahl C et al. Angular transmission response of In-plane symmetry-breaking quasi-BIC all-dielectric metasurfaces[J]. ACS Photonics, 9, 3642-3648(2022).
[29] Guo C, Xiao M, Minkov M et al. Photonic crystal slab Laplace operator for image differentiation[J]. Optica, 5, 251-256(2018).
[30] Cordaro A, Kwon H, Sounas D et al. High-index dielectric metasurfaces performing mathematical operations[J]. Nano Letters, 19, 8418-8423(2019).
[31] Zhou Y, Zheng H Y, Kravchenko I I et al. Flat optics for image differentiation[J]. Nature Photonics, 14, 316-323(2020).
[32] Fu W W, Zhao D, Li Z Q et al. Ultracompact meta-imagers for arbitrary all-optical convolution[J]. Light: Science & Applications, 11, 62(2022).
[33] Wang H W, Guo C, Zhao Z X et al. Compact incoherent image differentiation with nanophotonic structures[J]. ACS Photonics, 7, 338-343(2020).
[34] Zhang X M, Bai B F, Sun H B et al. Incoherent optoelectronic differentiation based on optimized multilayer films[J]. Laser & Photonics Reviews, 16, 2200038(2022).
[35] Zheng H Y, Liu Q, Zhou Y et al. Meta-optic accelerators for object classifiers[J]. Science Advances, 8, eabo6410(2022).
[36] Bernstein L, Sludds A, Panuski C et al. Single-shot optical neural network[J]. Science Advances, 9, eadg7904(2023).
[38] Wang X, Ma X, Ke J et al. Advances in speckle and compressive computational imaging[J]. Acta Optica Sinica, 43, 1511001(2023).
[39] Nyquist H. Certain topics in telegraph transmission theory[J]. Transactions of the American Institute of Electrical Engineers, 47, 617-644(1928).
[40] Gan L. Block compressed sensing of natural images[C], 403-406(2007).
[41] Yuan X, Brady D J, Katsaggelos A K. Snapshot compressive imaging: theory, algorithms, and applications[J]. IEEE Signal Processing Magazine, 38, 65-88(2021).
[42] Wagadarikar A A, Pitsianis N P, Sun X B et al. Spectral image estimation for coded aperture snapshot spectral imagers[J]. Proceedings of SPIE, 7076, 707602(2008).
[43] Gehm M E, John R, Brady D J et al. Single-shot compressive spectral imaging with a dual-disperser architecture[J]. Optics Express, 15, 14013-14027(2007).
[44] Wagadarikar A, John R, Willett R et al. Single disperser design for coded aperture snapshot spectral imaging[J]. Applied Optics, 47, B44-B51(2008).
[45] Veeraraghavan A, Reddy D, Raskar R. Coded strobing photography: compressive sensing of high speed periodic videos[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33, 671-686(2011).
[46] Holloway J, Sankaranarayanan A C, Veeraraghavan A et al. Flutter Shutter Video Camera for compressive sensing of videos[C](2012).
[47] Reddy D, Veeraraghavan A, Chellappa R. P2C2: Programmable pixel compressive camera for high speed imaging[C], 329-336(2011).
[48] Llull P, Liao X J, Yuan X et al. Coded aperture compressive temporal imaging[J]. Optics Express, 21, 10526-10545(2013).
[49] Koller R, Schmid L, Matsuda N et al. High spatio-temporal resolution video with compressed sensing[J]. Optics Express, 23, 15992-16007(2015).
[50] Yang P H, Kong L H, Liu X Y et al. Shearlet enhanced snapshot compressive imaging[J]. IEEE Transactions on Image Processing, 29, 6466-6481(2020).
[51] Liu Y, Yuan X, Suo J L et al. Rank minimization for snapshot compressive imaging[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 41, 2990-3006(2019).
[52] Boyd P S, Parikh N, Chu E et al. Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends® in Machine Learning, 3, 128(2010).
[53] Yuan X, Jiang H, Huang G et al. SLOPE: shrinkage of local overlapping patches estimator for lensless compressive imaging[J]. IEEE Sensors Journal, 16, 8091-8102(2016).
[54] Metzler C A, Maleki A, Baraniuk R G. From denoising to compressed sensing[J]. IEEE Transactions on Information Theory, 62, 5117-5144(2016).
[55] Yuan X, Llull P, Liao X J et al. Low-cost compressive sensing for color video and depth[C], 3318-3325(2014).
[56] Yuan X. Generalized alternating projection based total variation minimization for compressive sensing[C], 2539-2543(2016).
[57] Yang J B, Liao X J, Yuan X et al. Compressive sensing by learning a Gaussian mixture model from measurements[J]. IEEE Transactions on Image Processing, 24, 106-119(2015).
[58] Yang J B, Yuan X, Liao X J et al. Video compressive sensing using Gaussian mixture models[J]. IEEE Transactions on Image Processing, 23, 4863-4878(2014).
[59] Renna F, Wang L M, Yuan X et al. Classification and reconstruction of high-dimensional signals from low-dimensional features in the presence of side information[J]. IEEE Transactions on Information Theory, 62, 6459-6492(2016).
[60] Yuan X, Tsai T H, Zhu R Y et al. Compressive hyperspectral imaging with side information[J]. IEEE Journal of Selected Topics in Signal Processing, 9, 964-976(2015).
[61] Qiao M, Meng Z Y, Ma J W et al. Deep learning for video compressive sensing[J]. APL Photonics, 5, 030801(2020).
[62] Zhang B, Yuan X, Deng C et al. End-to-end snapshot compressed super-resolution imaging with deep optics[J]. Optica, 9, 451-454(2022).
[65] Cheng Z H, Lu R Y, Wang Z J, Vedaldi A, Bischof H, Brox T et al. BIRNAT: bidirectional recurrent neural networks with adversarial training for video snapshot compressive imaging[M]. Computer vision-ECCV 2020. Lecture notes in computer science, 12369, 258-275(2020).
[66] Cheng Z H, Chen B, Liu G L et al. Memory-efficient network for large-scale video compressive sensing[C], 16241-16250(2021).
[67] Li L G, Wang L Z, Song W T et al. Quantization-aware deep optics for diffractive snapshot hyperspectral imaging[C], 19748-19757(2022).
[68] Hu X W, Cai Y H, Lin J et al. HDNet: high-resolution dual-domain learning for spectral compressive imaging[C], 17521-17530(2022).
[69] Meng Z Y, Ma J W, Yuan X, Vedaldi A, Bischof H, Brox T et al. End-to-end low cost compressive spectral imaging with spatial-spectral self-attention[M]. Computer vision-ECCV 2020. Lecture notes in computer science, 12368, 187-204(2020).
[70] Cai Y H, Lin J, Hu X W et al. Mask-guided spectral-wise transformer for efficient hyperspectral image reconstruction[C], 17481-17490(2022).
[71] Cai Y H, Lin J, Hu X W, Avidan S, Brostow G, Cissé M et al. Coarse-to-fine sparse transformer for hyperspectral image reconstruction[M]. Computer vision-ECCV 2022. Lecture notes in computer science, 13677, 686-704(2022).
[76] Yang Y, Sun J, Li H B et al. Deep ADMM-Net for compressive sensing MRI[C], 10-18(2016).
[77] Ma J W, Liu X Y, Shou Z et al. Deep tensor ADMM-net for snapshot compressive imaging[C], 10222-10231(2020).
[78] Yuan X, Liu Y, Suo J L et al. Plug-and-play algorithms for large-scale snapshot compressive imaging[C], 1444-1454(2020).
[79] Wu Z, Zhang J, Mou C. Dense deep unfolding network with 3D-CNN prior for snapshot compressive imaging[C], 4872-4881(2022).
[80] Wang L Z, Sun C, Zhang M Q et al. DNU: deep non-local unrolling for computational spectral imaging[C], 1658-1668(2020).
[81] Wang L Z, Sun C, Fu Y et al. Hyperspectral image reconstruction using a deep spatial-spectral prior[C], 8024-8033(2020).
[82] Huang T, Dong W S, Yuan X et al. Deep Gaussian scale mixture prior for spectral compressive imaging[C], 16211-16220(2021).
[84] Wang P, Yuan X. SAUNet: spatial-attention unfolding network for image compressive sensing[C], 5099-5108(2023).
[85] Wu Z L, Yang C S, Su X F et al. Adaptive deep PnP algorithm for video snapshot compressive imaging[J]. International Journal of Computer Vision, 131, 1662-1679(2023).
[86] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 60, 84-90(2017).
[87] He K M, Zhang X Y, Ren S Q et al. Deep residual learning for image recognition[C], 770-778(2016).
[88] Vaswani A, Shazeer N, Parmar N et al. Attention is all you need[C], 6000-6010(2017).
[89] Goodfellow I, Pouget-Abadie J, Mirza M et al. Generative adversarial networks[J]. Communications of the ACM, 63, 139-144(2020).
[90] Zuo Y, Li B H, Zhao Y J et al. All-optical neural network with nonlinear activation functions[J]. Optica, 6, 1132-1137(2019).
[91] Bueno J, Maktoobi S, Froehly L et al. Reinforcement learning in a large-scale photonic recurrent neural network[J]. Optica, 5, 756-760(2018).
[92] Li J X, Mengu D, Luo Y et al. Class-specific differential detection in diffractive optical neural networks improves inference accuracy[J]. Advanced Photonics, 1, 046001(2019).
[93] Yan T, Wu J M, Zhou T K et al. Fourier-space diffractive deep neural network[J]. Physical Review Letters, 123, 023901(2019).
[94] Shen Y C, Harris N C, Skirlo S et al. Deep learning with coherent nanophotonic circuits[J]. Nature Photonics, 11, 441-446(2017).
[95] Zhu H H, Zou J, Zhang H et al. Space-efficient optical computing with an integrated chip diffractive neural network[J]. Nature Communications, 13, 1044(2022).
[96] Fu T Z, Zang Y B, Huang Y Y et al. Photonic machine learning with on-chip diffractive optics[J]. Nature Communications, 14, 70(2023).
[97] Cui T J, Qi M Q, Wan X et al. Coding metamaterials, digital metamaterials and programmable metamaterials[J]. Light: Science & Applications, 3, e218(2014).
[98] Liu C, Ma Q, Luo Z J et al. A programmable diffractive deep neural network based on a digital-coding metasurface array[J]. Nature Electronics, 5, 113-122(2022).
[99] Williamson I A D, Hughes T W, Minkov M et al. Reprogrammable electro-optic nonlinear activation functions for optical neural networks[J]. IEEE Journal of Selected Topics in Quantum Electronics, 26, 7700412(2020).
[100] Vandoorne K, Dierckx W, Schrauwen B et al. Toward optical signal processing using photonic reservoir computing[J]. Optics Express, 16, 11182-11192(2008).
[101] Vandoorne K, Dambre J, Verstraeten D et al. Parallel reservoir computing using optical amplifiers[J]. IEEE Transactions on Neural Networks, 22, 1469-1481(2011).
[102] Mesaritakis C, Papataxiarhis V, Syvridis D. Micro ring resonators as building blocks for an all-optical high-speed reservoir-computing bit-pattern-recognition system[J]. Journal of the Optical Society of America B, 30, 3048-3055(2013).
[103] Rosenbluth D, Kravtsov K, Fok M P et al. A high performance photonic pulse processing device[J]. Optics Express, 17, 22767-22772(2009).
[104] Li W, Liu X, Zhang W C et al. Space based edge intelligent remote sensing[J]. Spacecraft Recovery & Remote Sensing, 43, 1-11(2022).
[105] Zhang F, Chen X Q, Yu S et al. On-orbit artificial intelligence based on space edge computing[J]. Aerospace Shanghai (Chinese & English), 38, 19-24(2021).
[106] Gunasheela K S, Prasantha H S, Guru D S, Vasudev T, Chethan et al. Satellite image compression-detailed survey of the algorithms[M]. Proceedings of international conference on cognition and recognition. Lecture notes in computer science, 14, 187-198(2018).
[107] Hussain A J, Al-Fayadh A, Radi N. Image compression techniques: a survey in lossless and lossy algorithms[J]. Neurocomputing, 300, 44-69(2018).
[108] Liu S Y, Deng W H. Very deep convolutional neural network based image classification using small training sample size[C], 730-734(2016).
[110] Qian G W, Xu X Q, Zhu S et al. Quantum induced coherence light detection and ranging[J]. Physical Review Letters, 131, 033603(2023).
Get Citation
Copy Citation Text
Tianyu Li, Guoqing Wang, Wei Li, Hongwei Chen, Xun Liu, Zhibin Wang, Shaochong Liu, Yanxin Cai. Review of Optical Pre-Sensor Computing Technology and Its Satellite Remote Sensing Applications (Invited)[J]. Laser & Optoelectronics Progress, 2024, 61(2): 0211030
Category: Imaging Systems
Received: Nov. 15, 2023
Accepted: Dec. 13, 2023
Published Online: Feb. 6, 2024
The Author Email: Wang Guoqing (gqwang0420@uestc.edu.cn), Li Wei (wei_li_bj@163.com), Chen Hongwei (chenhw@tsinghua.edu.cn)