Chinese Journal of Liquid Crystals and Displays, Volume. 38, Issue 7, 975(2023)
Multimodal image semantic segmentation based on attention mechanism
Figures & Tables(8)
Fig. 3. Schematic diagram of two convolution block operations in the upsampling module
Fig. 4. Visual comparison of segmentation results of some network models
Table 1. Each module configuration in the decoder
View tableView in ArticleTable 1. Each module configuration in the decoder
模块 卷积块名称 卷积操作 Keneral size Stride padding 上采样模块 第一个转置卷积块 普通卷积 3×3 1 1 普通卷积 3×3 1 1 第二个转置卷积块 普通卷积 3×3 1 1 转置卷积 2×2 1 1 转置卷积 2×2 1 1 特征提取模块 普通卷积 3×3 1 1
Table 2. Comparison of results of serial network models on MFNet test set
View tableView in ArticleTable 2. Comparison of results of serial network models on MFNet test set
网络模型 汽车 行人 自行车 车道线 停车位 护栏 色锥 地面
凸起物
mAcc mIoU Acc IoU Acc IoU Acc IoU Acc IoU Acc IoU Acc IoU Acc IoU Acc IoU MFNet 77.2 65.9 67.0 58.9 53.9 42.9 36.2 29.9 19.1 9.9 0.8 8.5 30.3 25.2 30.0 27.7 45.1 39.7 FuseNet 81.0 75.6 75.2 66.3 64.5 51.9 51.0 37.8 28.7 15.0 0.0 0.0 31.4 21.4 51.9 45.0 52.4 45.6 DepthAwareCNN 85.2 77.0 61.7 53.4 76.0 56.5 40.2 30.9 9.9 29.3 22.8 6.4 32.9 30.1 36.5 32.3 55.1 46.1 RTFNet-152 93.0 87.4 79.3 70.3 76.8 62.7 60.7 45.3 38.5 29.8 0.0 0.0 45.5 29.1 74.7 55.7 63.1 53.2 FuseSeg-161 93.1 87.9 81.4 71.7 78.5 64.6 68.4 44.8 29.1 22.7 63.7 6.4 55.8 46.9 66.4 47.9 70.6 54.5 FEANet 93.3 87.8 82.7 71.1 76.7 61.1 65.5 46.5 26.6 22.1 70.8 6.6 66.6 55.3 77.3 48.9 73.2 55.3 本文方法 95.0 84.7 80.8 71.7 77.0 61.7 70.4 44.0 50.6 33.1 65.3 8.4 63.8 52.2 82.4 47.9 76.0 55.7
Table 3. Performance comparison of a series of models on a day-night test set
View tableView in ArticleTable 3. Performance comparison of a series of models on a day-night test set
网络模型 白天图像测试集 夜间图像测试集 mAcc mIoU mAcc mIoU MFNet 42.6 36.1 41.1 36.8 FuseNet 49.5 41.0 48.9 43.9 DepthAwareCNN 50.6 42.4 50.7 43.2 RTFNet-152 60.0 45.8 60.7 54.8 FuseSeg-161 62.1 47.8 67.3 54.6 FEANet 62.5 47.2 70.5 56.4 本文方法 67.2 47.3 73.3 58.4
Table 4. Control group experimental configuration details and results
View tableView in ArticleTable 4. Control group experimental configuration details and results
对照组名称 实验设置 评价指标 RGB编码器注意力模块 热红外图像注意力模块 编码器通过相加融合 编码器特征图通过拼接融合 解码器特征图通过相加融合 解码器特征图通过拼接融合 mACC mIOU 对照组A √ √ 72.00 53.50 对照组B √ √ √ 73.70 55.20 对照组C √ √ √ 70.50 55.10 对照组D √ √ √ √ 71.60 54.00 对照组E √ √ √ √ 62.20 49.00 本文网络 √ √ √ √ 76.00 55.70
Tools
Get Citation
Copy Citation Text
Ji-you ZHANG, Rong-fen ZHANG, Yu-hong LIU, Wen-hao YUAN. Multimodal image semantic segmentation based on attention mechanism[J]. Chinese Journal of Liquid Crystals and Displays, 2023, 38(7): 975
Paper Information
Category: Research Articles
Received: Sep. 16, 2022
Accepted: --
Published Online: Jul. 31, 2023
The Author Email: Rong-fen ZHANG (rfzhang@gzu.edu.cn)
© Copyright 2018-2021 | Chinese Laser Press.
All Rights Reserved 沪ICP备15018463号-20