Inter-Layer Interpolation Method of CT Images Combined with Feature Pyramid and Deformable Separable Convolution

Zhihong Hu; Xiaobao Liu; Tinqiang Yao; Jihong Shen

doi:10.3788/LOP231809

Laser & Optoelectronics Progress, Volume. 61, Issue 12, 1237004(2024)

Inter-Layer Interpolation Method of CT Images Combined with Feature Pyramid and Deformable Separable Convolution

Zhihong Hu¹, Xiaobao Liu^1、*, Tinqiang Yao¹, and Jihong Shen²

¹Facility of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650500, Yunnan , China

²First Affiliated Hospital of Kunming Medical University, Kunming 650093, Yunnan , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(14)

Fig. 1. Overall structure of the inter-layer interpolation network model

Download full size

Fig. 2. Structures of the MultiRes block and Res path. (a) MultiRes block; (b) Res path

Download full size

Fig. 3. Schematic diagrams of the standard convolution and DSepConv sampling position. (a) Standard convolution kernel; (b) DSepConv

Download full size

Fig. 4. Schematic diagram of the DSepConv module

Download full size

Fig. 5. Diagrams of triple groups samples

Download full size

Fig. 6. Comparison of the effects between the images generated by the model and the actual inter-layer images. (a) Actual inter-layer images; (b) images generated by the network model; (c) partially enlarged images; (d) difference visualization of the images

Download full size

Fig. 7. Visualization of the difference of the inter-layer images generated by different interpolation methods. (a) Proposed method; (b) RRIN; (c) SepConv; (d) RIFE; (e) AdaCof

Download full size

Fig. 8. Comparison of interpolation results of different interpolation methods. (a) Actual inter-layer images; (b) proposed method; (c) RRIN; (d) SepConv; (e) RIFE; (f) AdaCof

Download full size

Table 1. MultiResUNet architecture details

View table

Table 1. MultiResUNet architecture details

MultiRes block

Convolution layer（size）

Filter

Res path

Convolution layer（size）

Filter

MultiRes block 1

Conv2D（3×3）

Conv2D（1×1）

Res path 1

Conv2D（3×3）

Conv2D（1×1）

Conv2D（3×3）

Conv2D（1×1）

Conv2D（3×3）

Conv2D（1×1）

MultiRes block 2

MultiRes block 8

Conv2D（3×3）

Conv2D（1×1）

105

MultiRes block 3

MultiRes block 7

Conv2D（3×3）

Conv2D（1×1）

106

212

Res path 2

Conv2D（3×3）

Conv2D（1×1）

Conv2D（3×3）

Conv2D（1×1）

128

MultiRes block 4

MultiRes block 6

Conv2D（3×3）

Conv2D（1×1）

142

213

426

Res path 3

Conv2D（3×3）

Conv2D（1×1）

256

MultiRes block 5

Conv2D（3×3）

Conv2D（1×1）

142

248

427

853

—

Table 2. Relevant informations of the public datasets for experiment
View table
Table 2. Relevant informations of the public datasets for experiment
Dataset name Location Data type Image resolution /pixel Inter-layer distance /mm
TCGA-ESCA Abdomen and Lung CT（Dicom） 512×512 2.5 or 1.2
SPIE-AAPM Lung CT Challenge Lung CT（Dicom） 512×512 1.0

Table 3. Quantitative comparison results of each model in ablation experiment

View table

Table 3. Quantitative comparison results of each model in ablation experiment

Model	PSNR /dB	SSIM	LPIPS	Parameters / $10^{6}$	Memory size /MB
U-Net+SepConv	34.36	0.9373	0.0166	21.68	82.7
MultiResUNet+SepConv	34.46	0.9351	0.0147	7.51	28.7
U-Net+DSepConv	34.53	0.9363	0.0182	21.87	84.0
MultiResUNet+DSepConv	34.61	0.9362	0.0154	8.21	31.6
MultiResUNet+DSepConv+IEM	34.96	0.9381	0.0168	10.66	41.0

Table 4. Inter-layer interpolation results based on different loss functions with different weights

View table

Table 4. Inter-layer interpolation results based on different loss functions with different weights

Loss function	PSNR /dB	SSIM	LPIPS
$L_{c}$	32.29	0.9545	0.0187
$L_{c} + L_{p}$	33.92	0.9614	0.0091
$L_{c} + L_{a d v}$	33.21	0.9581	0.0083
$L_{c} + L_{p} + L_{a d v}$	33.85	0.9606	0.0086
$L_{c} + 0.1 L_{p} + 0.01 L_{a d v}$	34.25	0.9625	0.0081
$L_{c} + 0.01 L_{p} + 0.1 L_{a d v}$	34.17	0.9616	0.0074
$L_{c} + 0.01 L_{p} + 0.01 L_{a d v}$	34.48	0.9636	0.0077
$L_{c} + 0.01 L_{p} + 0.005 L_{a d v}$	34.76	0.9651	0.0079

Table 5. Inter-layer interpolation results under different learning rates
View table
Table 5. Inter-layer interpolation results under different learning rates
Learning rate PSNR /dB SSIM LPIPS
0.1000 33.74 0.9563 0.0096
0.0100 34.21 0.9625 0.0085
0.0010 34.76 0.9651 0.0079
0.0001 34.55 0.9637 0.0081

Table 6. Quantitative comparison results of different interpolation methods

View table

Table 6. Quantitative comparison results of different interpolation methods

Method	PSNR /dB	SSIM	LPIPS	Memory size /MB	Time /s
RRIN	33.0559	0.9257	0.0224	82.2	0.46
SepConv	33.4792	0.9293	0.0232	82.7	0.37
RIFE	33.6355	0.9334	0.0365	40.9	0.48
AdaCof	33.7786	0.9336	0.0409	83.3	0.43
Ours	34.1534	0.9365	0.0186	41.0	0.57

Tools

Get Citation

Copy Citation Text

Zhihong Hu, Xiaobao Liu, Tinqiang Yao, Jihong Shen. Inter-Layer Interpolation Method of CT Images Combined with Feature Pyramid and Deformable Separable Convolution[J]. Laser & Optoelectronics Progress, 2024, 61(12): 1237004

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites