Fusion of Cell Refractive Index and Bright-Field Micrographs Based on Convolutional Neural Networks

Zhongfa Liu; Yizhe Yang; Yu Fang; Xiaojing Wu; Siwei Zhu; Yong Yang

doi:10.3788/LOP202158.2217001

Laser & Optoelectronics Progress, Volume. 58, Issue 22, 2217001(2021)

Fusion of Cell Refractive Index and Bright-Field Micrographs Based on Convolutional Neural Networks

Zhongfa Liu^1,2, Yizhe Yang^1,2, Yu Fang^1,2, Xiaojing Wu^3、**, Siwei Zhu³, and Yong Yang^1,2、*

Author Affiliations

¹Institute of Modern Optics, Nankai University, Tianjin 300350, China

²Tianjin Key Laboratory of Micro-Scale Optical Information Science and Technology, Tianjin 300350, China

³Tianjin Union Medical Center, Institute of Translational Medicine, Nankai University, Tianjin 300121, China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(9)

$Schematic of graphene-based refractive index microscopy system$

Fig. 1. Schematic of graphene-based refractive index microscopy system

Download full size

$Analysis of principle and microscopic images. (a) Schematic of the principle of probe beam scanning to measure the refractive index of the cell; (b)microscopic image of cell refractive index obtained in experiment; (c) experimentally obtained bright-field micrograph of the cell$

Fig. 2. Analysis of principle and microscopic images. (a) Schematic of the principle of probe beam scanning to measure the refractive index of the cell; (b)microscopic image of cell refractive index obtained in experiment; (c) experimentally obtained bright-field micrograph of the cell

Download full size

Fig. 3. Fusion model FusionCNN

Download full size

Fig. 4. The framework of FusionCNN algorithm

Download full size

$Refractive index micrographs and bright-field images of three cells. (a)--(c) Refractive index micrographs of cell; (d)--(f) the corresponding cells bright-field images$

Fig. 5. Refractive index micrographs and bright-field images of three cells. (a)--(c) Refractive index micrographs of cell; (d)--(f) the corresponding cells bright-field images

Download full size

$Experimental results of fusion of refractive index micrographs and corresponding bright-field images of three groups of cells using GTF (gradient transfer fusion) method, WL (wavelet transform-based fusion) method, and FusionCNN (CNN algorithm-based fusion) method, respectively. (a)--(c) Original refractive index micrographs; (d)--(f) fusion results obtained using FusionCNN method; (g)--(i) fusion results obtained using GTF method; (j)--(l) fusion results obtained using WL method$

Fig. 6. Experimental results of fusion of refractive index micrographs and corresponding bright-field images of three groups of cells using GTF (gradient transfer fusion) method, WL (wavelet transform-based fusion) method, and FusionCNN (CNN algorithm-based fusion) method, respectively. (a)--(c) Original refractive index micrographs; (d)--(f) fusion results obtained using FusionCNN method; (g)--(i) fusion results obtained using GTF method; (j)--(l) fusion results obtained using WL method

Download full size

$Fusion of high spatial resolution bright-field image or low spatial resolution bright-field image with refractive index microscopic image. (a) Fusion using 700 pixel×700 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image; (b) fusion of 100 pixel×100 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image$

Fig. 7. Fusion of high spatial resolution bright-field image or low spatial resolution bright-field image with refractive index microscopic image. (a) Fusion using 700 pixel×700 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image; (b) fusion of 100 pixel×100 pixel bright-field image and 100 pixel×100 pixel refractive index microscopic image

Download full size

Table 1. Objective evaluation indicators^[21]

View table

Table 1. Objective evaluation indicators^[21]

Evaluation indicator	Formulaic expression	Physical significance
PSNR	$\frac{255^{2}}{\overset{M}{\sum_{i = 1}} \overset{N}{\sum_{j = 1}} {[F (i, j) - R (i, j)]}^{2}}$	Reflects the difference between two images at a specific pixel, the higher the peak signal-to-noise ratio, the closer it is to the ideal image and the better the result
ENT	$- \overset{L - 1}{\sum_{i = 0}} P_{i} lo g_{2} (P_{i})$	Reflects the amount of information in an image. The more information an image contains, the better the result
AG	$\frac{1}{MN} \overset{M}{\sum_{x = 1}} \overset{N}{\sum_{y = 1}} \sqrt[]{Δ f_{x}^{2} + Δ f_{y}^{2}}$	The larger the average gradient, the clearer the detail representation in the image

Table 2. Fusion performance comparison of different methods

View table

Table 2. Fusion performance comparison of different methods

Cell and fusion method		PSNR	ENT	AG
Cell 1	FusionCNN	24.7191	6.7251	0.0365
	GTF	24.2505	6.5551	0.0367
	WL	11.7205	6.3839	0.0283
Cell 2	FusionCNN	25.7789	6.3278	0.0314
	GTF	25.6525	6.1210	0.0303
	WL	11.4884	6.0491	0.0024
Cell 3	FusionCNN	26.1563	6.4638	0.0290
	GTF	26.0756	6.3835	0.0272
	WL	12.2216	6.4501	0.0206

Tools

Get Citation

Copy Citation Text

Zhongfa Liu, Yizhe Yang, Yu Fang, Xiaojing Wu, Siwei Zhu, Yong Yang. Fusion of Cell Refractive Index and Bright-Field Micrographs Based on Convolutional Neural Networks[J]. Laser & Optoelectronics Progress, 2021, 58(22): 2217001

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites