Extending Field‑of‑View of Two‑Photon Microscopy Using Deep Learning

Chijian Li; Jing Yao; Yufeng Gao; Puxiang Lai; Yuezhi He; Sumin Qi; Wei Zheng

doi:10.3788/CJL221433

Chinese Journal of Lasers, Volume. 50, Issue 9, 0907107(2023)

Extending Field‑of‑View of Two‑Photon Microscopy Using Deep Learning

Chijian Li^1,2, Jing Yao^2,3,4, Yufeng Gao², Puxiang Lai^3,4, Yuezhi He^2、***, Sumin Qi^1、**, and Wei Zheng^2、*

Author Affiliations

¹School of Cyber Science and Engineering, Qufu Normal University, Jining 273100, Shandong, China

²Research Center for Biomedical Optics and Molecular Imaging, Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen 518055, Guangdong, China

³Department of Biomedical Engineering, The Hong Kong Polytechnic University, Hong Kong SAR 999077, China

⁴Shenzhen Research Institute, The Hong Kong Polytechnic University, Shenzhen 518055, Guangdong, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(9)

Fig. 1. Optical systems and imaging principle. (a) Principle of extending field-of-view (FOV) of two-photon microscopy (TPM) using deep learning; (b) schematic of TPM system with large FOV (HWP: half wave plate; PBS: polarization splitting prism; PMT: photomultiplier; RM: reflective mirror; BE: beam expander; SLM: spatial light modulator; OAPM: off-axis parabolic mirror; SM: scan mirror; SL: scan lens; TL: tube lens; DM: dichroic mirror; OL: objective lens; CL: collective lens); (c) measured distort wavefronts of 3×3 subregions

Download full size

Fig. 2. Proposed nBRAnet network structure. (a) Improved three-layer network structure; (b) schematic of residual structure;

Download full size

Fig. 3. Large FOV imaging of fluorescent beads with diameter of 1 µm. (a) Image of fluorescent beads with the FOV of 2.45 mm×2.45 mm; (b)-(d) images with and without adaptive optics (AO) correction as well as obtained by network model learning in nominal FOV; (e)-(g) images with and without AO correction as well as obtained by network model learning in extended FOV; (h) intensity profiles of region Ⅰ; (i) intensity profiles of region Ⅱ

Download full size

Fig. 4. Large FOV imaging of brain slice of Thy1-GFP mouse. (a) Full FOV image with size of 2.45 mm×2.45 mm; (b) image of extended FOV in dash box before AO correction; (c) image of extended FOV in dash box after AO correction; (d) enhanced image obtained by deep learning model; (e)(f) intensity comparison along solid and dash lines

Download full size

Fig. 5. Large FOV imaging of microglia in CX3CR1-GFP mouse brain slice. (a) Full FOV image with size of 2.45 mm×2.45 mm; (b) image of extended FOV in dash box before AO correction; (c) image of extended FOV in dash box after AO correction; (d) enhanced image obtained by deep learning model; (e)-(g) histograms of gray values corresponding to Figs.(b)-(d), respectively

Download full size

Fig. 6. Output results of different network models. (a) ROI area of extended FOV of fluorescent bead samples; (b) ROI area of extended FOV of Thy1-GFP sample; (c) ROI area of extended FOV of CX3CR1-GFP sample

Download full size

Table 1. Applied Zernike polynomials

View table

Table 1. Applied Zernike polynomials

Index（i）	m	n	Zernike polynomial Z_i	Name
1	2	2	$\sqrt[]{6} r^{2} c o s (2 θ)$	1st，45°，astigmatism
2	2	-2	$\sqrt[]{6} r^{2} s i n (2 θ)$	1st，0°，astigmatism
3	3	3	$2 \sqrt[]{2} r^{3} c o s (3 θ)$	Trefoil
4	3	1	$2 \sqrt[]{2} (3 r^{2} - 2 r) c o s θ$	Coma X
5	3	-1	$2 \sqrt[]{2} (3 r^{2} - 2 r) s i n θ$	Coma Y
6	3	-3	$2 \sqrt[]{2} r^{3} s i n (3 θ)$	Trefoil
7	4	4	$\sqrt[]{10} r^{4} c o s (4 θ)$	Quadrafoil
8	4	2	$\sqrt[]{10} (4 r^{4} - 3 r^{2}) c o s (2 θ)$	2nd，45°，astigmatism
9	4	-2	$\sqrt[]{10} (4 r^{4} - 3 r^{2}) s i n (2 θ)$	2nd，0°，astigmatism
10	4	4	$\sqrt[]{10} r^{4} c o s (4 θ)$	Quadrafoil

Table 2. Results from ablation experiments for verifying BN structure’s effect
View table
Table 2. Results from ablation experiments for verifying BN structure’s effect
Sample BN or nBN Epoch Quantity of training set PSNR /dB
Fluorescent beads BN 150 25600 40.22
nBN 150 25600 40.39
Thy1-GFP BN 200 16000 26.93
nBN 200 16000 27.79
CX3CR1-GFP BN 200 16000 30.53
nBN 200 16000 31.37

Table 3. Experimental results evaluation of different network models

View table

Table 3. Experimental results evaluation of different network models

Network	PSNR /dB			Time /s			Model size （parameter quantity）	GFLOPs
Network	Fluorescent beads	Thy1-GFP	CX3CR1-GFP	Fluorescent beads	Thy1-GFP	CX3CR1-GFP	Model size （parameter quantity）	GFLOPs
U-Net	39.26	26.96	30.99	3.10	2.99	2.97	3.453×10⁷	833
VDSR	36.13	25.13	32.02	1.19	1.14	1.18	6.647×10⁵	666
Ours	40.39	27.56	31.37	4.10	4.06	4.33	3.558×10⁷	851

Tools

Get Citation

Copy Citation Text

Chijian Li, Jing Yao, Yufeng Gao, Puxiang Lai, Yuezhi He, Sumin Qi, Wei Zheng. Extending Field‑of‑View of Two‑Photon Microscopy Using Deep Learning[J]. Chinese Journal of Lasers, 2023, 50(9): 0907107

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites