Weakly supervised image semantic segmentation based on masked consistency mechanism

算法	弱监督语义分割
步骤1	训练MCM：
	输入：完整图像、对应被遮掩图像、被遮掩图像块的索引
	输出：类别、完整图像的类激活映射图、被遮掩图像的类激活映射图
	标签：类别
	损失：分类损失、类激活映射图的正则化损失
	使用MCM模型输出训练集数据的类激活映射图。
步骤2	训练AffinityNet：
	输入：训练图像
	输出：特征图
	标签：相邻像素对的二进制语义标签
	损失：目标对和背景对的交叉熵损失
	使用AffinityNet修正训练集的类激活映射图，AffinityNet生成转移概率矩阵，使用随机游走策略感知语义边界并生成伪掩码标签。
步骤3	训练Deeplab v1：
	输入：训练图像
	输出：预测掩码
	标签：伪掩码
	损失：平均交并比损失
	Deeplab v1预测验证集的语义分割掩码，与分割标签计算mIoU用于评估。

Table 2. Evaluation of initial seed (Seed) and pseudo segmentation label (Mask) in terms of ground truth on PASCAL VOC 2012 training set
View table
View in Article
Table 2. Evaluation of initial seed (Seed) and pseudo segmentation label (Mask) in terms of ground truth on PASCAL VOC 2012 training set
方法 Seed/% Mask/%
SEAM^[9] 55.4 63.6
AdvCAM^[21] 55.6 68.0
TS-CAM^[13] 40.3 -
MCTformer^[14] 61.7 69.1
RIB^[22] 56.5 68.6
MCM 62.0 70.8

Table 3. Comparison with state-of-the-art WSSS method on PASCAL VOC 2012 validation set

View table

View in Article

Table 3. Comparison with state-of-the-art WSSS method on PASCAL VOC 2012 validation set

方法	骨干网络	监督信号	mIoU/%
CIAN^[3]	ResNet101	I+S	64.3
ICD^[23]	ResNet101	I+S	67.8
EDAM^[24]	ResNet101	I+S	70.9
EPS^[2]	ResNet101	I+S	71.0
L2G^[6]	ResNet101	I+S	72.1
AuxSegNet^[25]	ResNet38	I+S	69.0
SEAM^[9]	ResNet38	I	64.5
BES^[26]	ResNet101	I	65.7
CONTA^[27]	ResNet38	I	66.1
AdvCAM^[21]	ResNet101	I	68.1
ECS-Net^[28]	ResNet38	I	66.6
CDA^[29]	ResNet38	I	66.1
MCTformer^[14]	ResNet38	I	71.9
PMM^[30]	ResNet38	I	68.5
URN^[31]	ResNet38	I	69.4
URN^[31]	ResNet101	I	69.5
AMN^[32]	ResNet101	I	70.7
MCM	ResNet38	I	73.1

Table 4. Comparison with state-of-the-art WSSS method on MS COCO 2014 validation set

View table

View in Article

Table 4. Comparison with state-of-the-art WSSS method on MS COCO 2014 validation set

方法	骨干网络	监督信号	mIoU/%
EPS^[2]	ResNet101	I+S	35.7
AuxSegNet^[25]	ResNet38	I+S	33.9
L2G^[6]	ResNet101	I+S	44.2
SEAM^[9]	ResNet38	I	31.9
CONTA^[27]	ResNet38	I	32.8
CDA^[29]	ResNet38	I	33.2
PMM^[30]	ResNet38	I	36.7
URN^[31]	ResNet38	I	40.5
MCTformer^[14]	ResNet38	I	42.0
AMN^[32]	ResNet101	I	44.7
URN^[31]	ResNet101	I	40.7
RIB^[22]	ResNet101	I	43.8
MCM	ResNet38	I	43.7

Table 5. Ablation study for each loss function

View table

View in Article

Table 5. Ablation study for each loss function

方法	PASCAL VOC 2012 验证集 (mIoU/%)	MS COCO 2014 验证集 (mIoU/%)
基准(MCTformer+ $l_{cls - c}$ )	71.9	42.0
MCM( $l_{cls - c}$ + $l_{cls - inc}$ )	72.1 (+0.2)	42.3 (+0.3)
MCM( $l_{cls - c}$ + $l_{cls - inc}$ + $l_{MC}$ )	72.1 (+0.2)	42.4 (+0.4)
MCM( $l_{cls - c}$ + $l_{cls - inc}$ +10 $l_{MC}$ )	73.1 (+1.2)	43.7 (+1.7)
MCM( $l_{cls - c}$ + $l_{cls - inc}$ +50 $l_{MC}$ )	72.7 (+0.8)	43.5 (+1.5)

Table 6. Ablation study for masking methods
View table
View in Article
Table 6. Ablation study for masking methods
方法 PASCAL VOC 2012验证集(mIoU/%)
MCTformer 71.9
MCM 73.1 (+1.2)
MCM^† 72.0 (+0.1)

Tools

Get Citation

Copy Citation Text

Jie HU, Haitao ZHAO. Weakly supervised image semantic segmentation based on masked consistency mechanism[J]. Journal of Applied Optics, 2024, 45(4): 741

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Research Articles

Received: Jun. 15, 2023

Accepted: --

Published Online: Oct. 21, 2024

The Author Email: ZHAO Haitao (赵海涛)

DOI:10.5768/JAO202445.0402003

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Flow chart of weakly supervised semantic segmentation algorithm

Table 1. Flow chart of weakly supervised semantic segmentation algorithm

Table 2. Evaluation of initial seed (Seed) and pseudo segmentation label (Mask) in terms of ground truth on PASCAL VOC 2012 training set

Table 2. Evaluation of initial seed (Seed) and pseudo segmentation label (Mask) in terms of ground truth on PASCAL VOC 2012 training set

Table 3. Comparison with state-of-the-art WSSS method on PASCAL VOC 2012 validation set

Table 3. Comparison with state-of-the-art WSSS method on PASCAL VOC 2012 validation set

Table 4. Comparison with state-of-the-art WSSS method on MS COCO 2014 validation set

Table 4. Comparison with state-of-the-art WSSS method on MS COCO 2014 validation set

Table 5. Ablation study for each loss function

Table 5. Ablation study for each loss function

Table 6. Ablation study for masking methods

Table 6. Ablation study for masking methods

微信扫一扫：分享