Advanced Photonics, Volume. 5, Issue 6, 066003(2023)

Harnessing the magic of light: spatial coherence instructed swin transformer for universal holographic imaging

Xin Tong1,2, Renjun Xu2, Pengfei Xu1, Zishuai Zeng1, Shuxi Liu1, and Daomu Zhao1、*
Author Affiliations
  • 1Zhejiang University, School of Physics, Zhejiang Province Key Laboratory of Quantum Technology and Device, Hangzhou, China
  • 2Zhejiang University, Center for Data Science, Hangzhou, China
  • show less
    Figures & Tables(10)
    Principle and performance of TWC-Swin method. (a) LPR. SC modulation can adjust the SC by changing the distance D. Holographic modulation is used to load the phase hologram. The LPR generates two outputs, one for calculating SC and the other for network input. HWP, half-wave plate; PBS, polarized beam splitter; L, lens; RD, rotating diffuser; SLM, spatial light modulator; F, filter. D, distance between L1 and RD. (b) The detailed flow of the TWC-Swin method. The swin adapter can select the optimal model from the model space by obtaining SC. The color picture represents a case in progress. (c) Swin-model space and architecture of the swin model. The architecture of M1−M11 is the same; only the weights are different. The weights are obtained by network training at different distances. (d) The correspondence between SC and swin-model space. See Table S1 in the Supplementary Material for detailed data. (e) Inputs and outputs of the swin model with different SCs. (f) SSIM and PCC of swin-model outputs at different SCs. (g) Training and test data acquisition process. The training data did not contain any turbulence. (h) SSIM and PCC of swin-model outputs at different turbulent scenes.
    Qualitative analysis of our method’s performance at the different SCs. Input, raw image captured by CMOS1. Output, image processed by the network. (a)–(k) Different SCs: (a) D=f1, SC is 0.494; (b) D=1.1f1, SC is 0.475; (c) D=1.2f1, SC is 0.442; (d) D=1.3f1, SC is 0.419; (e) D=1.4f1, SC is 0.393; (f) D=1.5f1, SC is 0.368; (g) D=1.6f1, SC is 0.337; (h) D=1.7f1, SC is 0.311; (i) D=1.8f1, SC is 0.285; (j) D=1.9f1, SC is 0.25; and (k) D=2f1, SC is 0.245. D means the distance between L1 and RD in the LPR and f1 is the focal length of L1. Our method can achieve improved image quality under low SC (Video 1, MP4, 1.5 MB [URL: https://doi.org/10.1117/1.AP.5.6.066003.s1]).
    Average results of the evaluation indices for each test data set. The coherence is 0.368. Results of other coherences are provided in Fig. S2 in the Supplementary Material. All evaluation indices demonstrate that our method possesses strong image restoration ability under low SC.
    Qualitative analysis of our method’s performance across varying intensities of (a) oceanic and (b) atmospheric turbulence. The network trained with coherence as physical prior information can effectively overcome the impact of turbulence on imaging and improve image quality. (O1)–(O5) mean oceanic turbulence phase and (A1)–(A5) mean atmospheric turbulence phase. (O1) χt=10−9 K2/s, coherence is 0.491. (O2) χt=10−7 K2/s, coherence is 0.482. (O3) χt=2×10−7 K2/s, coherence is 0.447. (O4) χt=4×10−7 K2/s, coherence is 0.404. (O5) χt=10−6 K2/s, coherence is 0.373. (A1) Cn2=10−14 m3−α, coherence is 0.507. (A2) Cn2=1.5×10−13 m3−α, coherence is 0.459. (A3) Cn2=2.5×10−13 m3−α, coherence is 0.43. (A4) Cn2=3.5×10−13 m3−α, coherence is 0.403. (A5) Cn2=5×10−13 m3−α, coherence is 0.378. Other parameter settings of the turbulent power spectrum function can be found in Table S2 in the Supplementary Material (Video 2, MP4, 36.4 MB [URL: https://doi.org/10.1117/1.AP.5.6.066003.s2]).
    Visualization of performance of different methods. The SSIM is shown in the bottom left corner. Our method presents the best performance, which is shown by smoother images with lower noise. (a) Sample selected with the WED data set and magnified insets of the red bounding region. (b) Sample selected with Flickr data set and magnified insets of the red bounding region. The pure swin model can be obtained by removing the postprocessing block of the swin model (Video 3, MP4, 0.6 MB [URL: https://doi.org/10.1117/1.AP.5.6.066003.s3]).
    Performance between different methods on various data sets with SC being 0.494. Our model outperforms other methods across various data sets and indices.
    (a), (b) Performance comparison between different methods at various turbulent scenes. (A1) Cn2=10−14 m3−α, coherence is 0.506. (A2) Cn2=1.5×10−13 m3−α, coherence is 0.459. (O1) χt=10−9 K2/s, coherence is 0.491. (O2) χt=10−7 K2/s, coherence is 0.482. Note that all methods are trained with coherence as physical prior information and improve image quality under turbulence conditions. This demonstrates that incorporating appropriate physical prior information can help the network cope with multiscene tasks.
    • Table 1. Quantitative analysis of evaluation indices (SSIM and PCC) at different SCs and test samplesa. f1 is the focal length of L1. SC means spatial coherence of the light source.

      View table
      View in Article

      Table 1. Quantitative analysis of evaluation indices (SSIM and PCC) at different SCs and test samplesa. f1 is the focal length of L1. SC means spatial coherence of the light source.

      SCSSIMPCC
      BSDCelebAFlickrWEDDIVBSDCelebAFlickrWEDDIV
      Input_f1, SC = 0.4940.58930.59430.42960.61550.46250.93680.95750.92100.91460.8753
      Output_f10.89840.89080.85230.90190.89400.98070.98930.98480.99300.9819
      Input_1.3f1, SC = 0.4190.57750.54150.39170.62450.41840.89530.93030.85880.91490.8043
      Output_1.3f10.91890.88420.86760.89970.89180.98430.99280.98800.99280.9827
      Input_1.5f1, SC = 0.3680.61780.53940.27770.56770.38920.89570.92110.83960.89610.8144
      Output_1.5f10.89060.85130.81710.85410.86220.96910.98810.97830.98690.9680
      Input_1.7f1, SC = 0.3110.60400.50170.31830.55100.41360.83030.90350.85110.85680.7979
      Output_1.7f10.86240.77910.74830.80130.80380.96440.97870.97020.97590.9583
      Input_2f1, SC = 0.2450.48810.44690.30730.52710.36430.80720.88170.75570.83260.7196
      Output_2f10.81460.75400.69620.77220.75720.94310.97130.95050.96310.9341
      Ground truth1111111111
    • Table 2. Quantitative analysis of evaluation indices (SSIM and PCC) at different oceanic turbulence intensitiesa.

      View table
      View in Article

      Table 2. Quantitative analysis of evaluation indices (SSIM and PCC) at different oceanic turbulence intensitiesa.

      Oceanic turbulenceSSIMPCC
      BSDCelebAFlickrWEDDIVBSDCelebAFlickrWEDDIV
      Input (O1)0.53310.67730.68100.60160.70180.89780.94040.88760.90960.8718
      Output (O1)0.80880.79160.83680.80770.81720.93030.97070.93340.95600.9044
      Input (O2)0.50980.65660.66900.57160.53710.88550.93290.87860.89700.8494
      Output (O2)0.78230.76090.80150.78190.80050.92110.96110.92090.94480.8901
      Input (O3)0.49500.65380.65750.54550.52810.87640.93130.85850.89160.8371
      Output (O3)0.71910.71690.84340.73780.79840.88960.94130.88710.93440.8793
      Input (O4)0.47960.64080.64740.50340.50740.87740.92450.85760.86640.8130
      Output (O4)0.70600.69320.72870.67180.72170.88470.93790.88350.88920.8213
      Input (O5)0.45190.60410.62020.44460.49450.84560.90750.82870.82810.7631
      Output (O5)0.68990.67210.72250.62860.69580.89090.94150.88880.88390.8152
      Ground truth1111111111
    • Table 3. Quantitative analysis of evaluation indices (SSIM and PCC) at different atmospheric turbulence intensitiesa.

      View table
      View in Article

      Table 3. Quantitative analysis of evaluation indices (SSIM and PCC) at different atmospheric turbulence intensitiesa.

      Atmospheric turbulenceSSIMPCC
      BSDCelebAFlickrWEDDIVBSDCelebAFlickrWEDDIV
      Input (A1)0.57380.68210.69880.64950.63380.90140.94040.89290.91600.9766
      Output (A1)0.77980.77410.83370.81610.82310.93610.95640.92150.95740.9116
      Input (A2)0.53110.65130.67270.57430.57010.87970.92640.86760.88960.8279
      Output (A2)0.73120.69380.76990.69600.75810.89200.93530.89240.91410.8643
      Input (A3)0.50830.63830.67850.53480.57200.86880.92020.84930.87470.8081
      Output (A3)0.66150.67970.74270.63620.73690.88430.93920.87080.89190.8418
      Input (A4)0.49650.62640.66350.52020.55750.85900.91610.83640.86730.8040
      Output (A4)0.69150.67510.72870.63360.72730.87890.93080.87050.88550.8331
      Input (A5)0.49590.61530.65950.48400.54070.85240.90800.82630.84930.7862
      Output (A5)0.67610.68930.72010.61270.68020.87190.94650.88750.87490.8255
      Ground truth1111111111
    Tools

    Get Citation

    Copy Citation Text

    Xin Tong, Renjun Xu, Pengfei Xu, Zishuai Zeng, Shuxi Liu, Daomu Zhao, "Harnessing the magic of light: spatial coherence instructed swin transformer for universal holographic imaging," Adv. Photon. 5, 066003 (2023)

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Research Articles

    Received: Jul. 10, 2023

    Accepted: Sep. 26, 2023

    Posted: Sep. 26, 2023

    Published Online: Jan. 4, 2024

    The Author Email: Zhao Daomu (dmz123@zju.edu.cn)

    DOI:10.1117/1.AP.5.6.066003

    Topics