Multi-Level Branch Cross-Scale Fusion Network for High-Precision Semantic Segmentation in Complex Remote Sensing Environments

Junying Zeng; Senyao Deng; Chuanbo Qin; Yikui Zhai; Xudong Jia; Yajin Gu; Jiahua Xu

doi:10.3788/LOP241148

Laser & Optoelectronics Progress, Volume. 62, Issue 4, 0428003(2025)

Multi-Level Branch Cross-Scale Fusion Network for High-Precision Semantic Segmentation in Complex Remote Sensing Environments

Junying Zeng^*, Senyao Deng, Chuanbo Qin, Yikui Zhai, Xudong Jia, Yajin Gu, and Jiahua Xu

Author Affiliations

School of Electronics and Information Engineering, Wuyi University, Jiangmen 529020, Guangdong , China

show less

Abstract Get PDF(in Chinese)

Figures & Tables(20)

Fig. 1. Architecture of MBCFNet

Download full size

Fig. 2. Details of the operation of the shallow Swin Transformer

Download full size

Fig. 3. Spatial squeeze module

Download full size

Fig. 4. Spatial refinement module

Download full size

Fig. 5. Cross-scale fusion module

Download full size

Fig. 6. Multi-scale decoding module

Download full size

Fig. 7. Semantic segmentation prediction maps of different models on the Vaihingen dataset

Download full size

Fig. 8. Semantic segmentation prediction maps of different models on the Potsdam dataset

Download full size

Fig. 9. Semantic segmentation prediction maps of different models on the Uavid dataset

Download full size

Fig. 10. Confusion matrix comparison between MBCFNet and the suboptimal model on three remote sensing datasets

Download full size

Fig. 11. Visualization of the ablation experiments for different branch fusions

Download full size

Fig. 12. Visualization of the ablation experiments for different shallow networks

Download full size

Fig. 13. Visualization of the ablation experiments for the proposed modules

Download full size

Table 1. Parameters of the convolutional layers in the semantic branch

View table

Table 1. Parameters of the convolutional layers in the semantic branch

Layer name	Detail	Dimension
Layer 1	$\begin{array}{l} [C o n v 7 \times 7, s t r i d e i s 2] \\ [\begin{matrix} C o n v 3 \times 3 \\ C o n v 3 \times 3 \\ C o n v 1 \times 1 \end{matrix}] \times 4 \end{array}$	$\frac{H}{4} \times \frac{W}{4} \times 2 C$
Layer 2	$\begin{array}{l} [C o n v 5 \times 5, s t r i d e i s 2] \\ [\begin{matrix} C o n v 3 \times 3 \\ C o n v 3 \times 3 \\ C o n v 1 \times 1 \end{matrix}] \times 6 \end{array}$	$\frac{H}{8} \times \frac{W}{8} \times 4 C$
Layer 3	$[\begin{matrix} C o n v 3 \times 3, s t r i d e i s 2 \\ C o n v 3 \times 3 \\ C o n v 1 \times 1 \end{matrix}]$	$\frac{H}{16} \times \frac{W}{16} \times 8 C$
Layer 4	$[\begin{matrix} C o n v 3 \times 3, s t r i d e i s 2 \\ C o n v 3 \times 3 \\ C o n v 1 \times 1 \end{matrix}]$	$\frac{H}{32} \times \frac{W}{32} \times 16 C$
Layer 5	$[\begin{matrix} C o n v 3 \times 3, s t r i d e i s 2 \\ C o n v 3 \times 3 \\ C o n v 1 \times 1 \end{matrix}]$	$\frac{H}{64} \times \frac{W}{64} \times 32 C$

Table 2. Comparative results of different models on the Vaihingen dataset

View table

Table 2. Comparative results of different models on the Vaihingen dataset

Method	Class IoU						mIoU	mF1	OA	mPrecision	mRecall
Method	back.clu.	car	tree	low veg.	build.	imp. sur.	mIoU	mF1	OA	mPrecision	mRecall
MBCFNet	91.98	76.09	87.35	82.74	93.73	89.68	86.93	91.58	93.35	91.72	91.44
U-Net	63.52	75.57	85.59	81.54	93.69	89.44	81.56	86.75	88.69	86.95	86.55
DeepLabV3+	86.12	66.92	83.95	78.21	92.75	86.80	82.46	88.35	90.25	88.51	88.19
BiSeNetV1	91.80	67.62	82.84	76.83	92.32	86.28	82.95	88.94	91.25	89.33	88.55
CCNet	71.10	75.07	84.27	78.74	92.56	86.77	81.42	86.55	88.44	86.87	86.23
Mit-B2	86.97	69.25	85.18	81.15	92.48	89.64	84.21	89.91	91.92	90.28	89.54
ST-UNet	67.78	75.38	81.22	75.46	90.69	85.25	79.30	84.92	85.37	85.66	84.19
GLOTS	87.60	63.66	84.79	80.01	93.63	88.99	83.11	89.46	91.39	89.53	89.39

Table 3. Comparative results of different models on the Potsdam dataset

View table

Table 3. Comparative results of different models on the Potsdam dataset

Method	Class IoU						mIoU	mF1	OA	mPrecision	mRecall
Method	back. clu.	car	tree	low veg.	build.	imp. surf.	mIoU	mF1	OA	mPrecision	mRecall
MBCFNet	63.88	86.23	85.95	85.96	94.22	90.82	84.51	89.43	91.35	90.05	88.82
U-Net	45.61	82.70	81.70	80.93	89.80	87.62	78.06	83.85	87.77	83.93	83.77
DeepLabV3+	55.15	83.87	81.46	79.71	93.00	87.58	80.13	86.19	88.41	87.13	85.27
BiSeNetV1	52.83	82.46	81.77	81.19	92.69	88.07	79.84	85.95	88.35	86.95	84.97
CCNet	52.12	83.47	80.04	80.83	94.19	87.35	79.67	85.78	88.31	86.84	84.74
Mit-B2	58.38	84.82	82.56	82.56	93.69	88.97	81.83	87.50	89.12	87.77	87.23
ST-UNet	62.85	81.18	77.47	78.45	93.56	87.92	80.24	86.55	88.53	87.51	85.61
GLOTS	63.82	80.56	79.79	77.63	93.67	85.91	80.23	86.48	88.49	87.46	85.52

Table 4. Comparative results of different models on the Uavid dataset

View table

Table 4. Comparative results of different models on the Uavid dataset

Method	Class IoU								mIoU	mF1	OA	mPrecision	mRecall
Method	clu.	build.	road	tree	low veg.	mov. car	sta. car	hum.	mIoU	mF1	OA	mPrecision	mRecall
MBCFNet	71.41	86.11	79.27	83.47	73.70	77.47	69.88	55.10	74.55	81.32	86.70	82.05	80.60
U-Net	55.67	77.54	61.93	74.56	49.85	68.31	52.18	53.11	61.64	69.17	75.15	69.43	68.91
DeepLabV3+	64.49	84.12	73.48	79.57	61.69	74.35	58.50	54.14	68.79	74.62	81.23	75.63	73.63
BiSeNetV1	60.65	84.67	73.87	78.34	59.43	70.33	53.45	52.66	66.68	73.63	81.02	74.85	72.45
CCNet	59.64	83.75	72.53	76.97	57.07	68.75	53.52	52.53	65.60	72.59	80.10	73.00	72.18
Mit-B2	69.61	85.24	82.74	83.01	71.98	70.09	62.25	28.32	69.16	76.66	81.87	77.06	76.26
ST-UNet	64.29	81.52	74.02	74.43	64.78	76.94	68.94	59.09	70.50	77.19	82.23	77.56	76.82
GLOTS	70.92	84.00	83.64	82.83	72.65	74.76	68.32	50.89	73.50	79.71	84.94	80.50	78.94

Table 5. Ablation results of different branch fusions on three remote sensing datasets

View table

Table 5. Ablation results of different branch fusions on three remote sensing datasets

Dataset	Main	Boundary	Spatial	mIoU	mF1	OA
Vaihingen	√	×	×	84.79	90.13	90.23
	√	√	×	85.16	90.26	91.42
	√	×	√	85.24	90.33	91.46
	√	√	√	86.93	91.58	93.35
Potsdam	√	×	×	80.88	86.73	88.51
	√	√	×	81.50	87.11	89.03
	√	×	√	82.39	87.81	89.39
	√	√	√	84.51	89.43	91.35
Uavid	√	×	×	71.25	77.63	83.81
	√	√	×	72.54	78.39	84.59
	√	×	√	73.05	78.89	84.92
	√	√	√	74.55	81.32	86.70

Table 6. Ablation results of different shallow networks on three remote sensing datasets

View table

Table 6. Ablation results of different shallow networks on three remote sensing datasets

Dataset	Shallow network	mIoU	mF1	OA
Vaihingen	ResNet	83.55	89.67	91.63
	Mit	85.81	90.66	91.85
	Swin Transformer	86.93	91.58	93.35
Potsdam	ResNet	81.19	86.98	88.99
	Mit	82.31	87.63	89.16
	Swin Transformer	84.51	89.43	91.35
Uavid	ResNet	71.19	77.58	83.05
	Mit	73.28	79.18	85.51
	Swin Transformer	74.55	81.32	86.70

Table 7. Ablation results of the proposed modules on the three remote sensing datasets

View table

Table 7. Ablation results of the proposed modules on the three remote sensing datasets

Dataset	Module		mIoU	mF1	OA
Dataset	Cross-scale fusion	Multi-scale decoding	mIoU	mF1	OA
Vaihingen	×	×	82.27	88.25	88.70
	√	×	86.29	91.08	92.46
	×	√	85.88	90.84	91.93
	√	√	86.93	91.58	93.35
Potsdam	×	×	81.02	86.82	88.87
	√	×	83.69	88.80	90.25
	×	√	83.15	88.59	89.78
	√	√	84.51	89.43	91.35
Uavid	×	×	70.07	77.06	82.12
	√	×	74.02	80.89	86.51
	×	√	73.69	79.51	85.68
	√	√	74.55	81.32	86.70

Tools

Get Citation

Copy Citation Text

Junying Zeng, Senyao Deng, Chuanbo Qin, Yikui Zhai, Xudong Jia, Yajin Gu, Jiahua Xu. Multi-Level Branch Cross-Scale Fusion Network for High-Precision Semantic Segmentation in Complex Remote Sensing Environments[J]. Laser & Optoelectronics Progress, 2025, 62(4): 0428003

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Remote Sensing and Sensors

Received: Apr. 22, 2024

Accepted: Jul. 10, 2024

Published Online: Feb. 10, 2025

The Author Email:

DOI:10.3788/LOP241148

CSTR:32186.14.LOP241148

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology