Tactile-assisted point cloud super-resolution

Haoran Shen; Puzheng Wang; Ming Lu; Chi Zhang; Jian Li; Qin Wang

doi:10.3788/COL202523.051102

Chinese Optics Letters, Volume. 23, Issue 5, 051102(2025)

Tactile-assisted point cloud super-resolution

Haoran Shen^1,2, Puzheng Wang^1,2, Ming Lu^1,2, Chi Zhang^1,2, Jian Li^1,2、**, and Qin Wang^1,2、*

¹Institute of Quantum Information and Technology, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

²Broadband Wireless Communication and Sensor Network Technology, Key Lab of Ministry of Education, Nanjing University of Posts and Telecommunications, Nanjing 210003, China

show less

Figures & Tables(12)

Fig. 1. An illustration of the tactile-assisted framework. Given sparse point cloud $P$ with N points and touch point cloud $T$ with N points, the feature extraction block extracts feature maps F_p and F_t from the input, then feeds them into the feature fusion block, where F_t and F_p are merged to produce the fused feature map F_f. Next, the transformer encoder consumes both $P$ and the fused feature F_f to refine the feature map, then a high-resolution point cloud Q is obtained through coordinate reconstruction.

Download full size

View in Article

Fig. 2. The architecture of the feature extraction block (FE block).

Download full size

View in Article

Fig. 3. The architecture of the feature fusion block (FF block). In this module, we iteratively fuse tactile features F_t into visual features F_p, ultimately obtaining the fused features F_f. Specifically, during the first fusion of tactile features, the input features are the initial point cloud features F_p.

Download full size

View in Article

Fig. 4. An object from the TSR-PD, where (a) represents the high-resolution point cloud (GT), (b) corresponds to the low-resolution point cloud (blue) and tactile information (red) for 5 touches, and (c) depicts the point cloud from one tactile interaction.

Download full size

View in Article

Fig. 5. Comparing point set upsampling (16×) results from sparse inputs with and without tactile information using 512 input points. Among them are (a) joint, (b) arch, and (c) lamp post. The first row is the input low-resolution point cloud, the second row is the reconstructed point cloud without tactile information, the third row is the reconstructed point cloud with tactile information, and the fourth row is GT.

Download full size

View in Article

Fig. 6. Visualization results of different algorithms for upsampling on the same objects (a). We show the 16× upsampled results of (b) input point clouds (512 points) when processed by different upsampling methods: (c) PU-GCN^[28], (d) Grad-PU^[43], (e) PU-Transformer^[29], and (f) TAPSR.

Download full size

View in Article

Table 1. Feature Fusion Pipeline

View table

View in Article

Table 1. Feature Fusion Pipeline

Require: a low-resolution point cloud feature

f_{p}

touch point cloud features

f_{t i}

i = {1,2,…, α}

Ensure: a fused feature

f_{f}

1: for

i \in {1,2,…, α}

2: if

i = = 1

then

f_{i} = f_{p}

4: end if

f_{i + 1} = FeaFus (f_{p}, f_{t i}, f_{i})

6: end for

f_{f} = f_{i + 1}

8: return

f_{f}

Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Method^a
View table
View in Article
Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Method^a
Number of touches 0 1 2 3 4 5
CD 1.162 0.953 0.791 0.716 0.671 0.778
HD 3.724 3.484 3.313 3.312 3.291 3.314
EMD 5.421 5.079 5.064 5.099 5.099 5.092

Table 2. Quantitative Comparisons to Other Methods on the TSR-PD^a
View table
View in Article
Table 2. Quantitative Comparisons to Other Methods on the TSR-PD^a
CD HD EMD
PU-GAN^[27] 4.634 9.219 13.672
PU-GCN^[28] 3.009 8.751 10.576
Grad-PU^[43] 2.464 6.308 9.582
PU-Transformer^[29] 1.162 3.724 5.421
Ours (Number of touch = 4) 0.671 3.291 5.099

Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Work^a
View table
View in Article
Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Work^a
Rate PU-Transformer^[29] Ours
8× 1.096 0.832
16× 1.162 0.671
32× 1.184 0.895

Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)^a
View table
View in Article
Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)^a
FE block FF block Number of touches
0 1 2 3 4 5
× × 1.162 1.242 1.271 1.220 1.253 1.256
✓ × — 1.018 0.858 1.123 1.094 1.117
✓ ✓ — 0.953 0.791 0.716 0.671 0.778

Table 5. Training Speed and Inference Time for the Model With Different Numbers of Touches (r = 16)
View table
View in Article
Table 5. Training Speed and Inference Time for the Model With Different Numbers of Touches (r = 16)
Number of touches Training speed (per epoch) Inference time (per sample)
0 76.68 s 15.8 ms
1 78.35 s 23.9 ms
2 79.39 s 24.0 ms
3 80.35 s 24.1 ms
4 82.18 s 24.1 ms
5 83.45 s 24.2 ms

Tools

Get Citation

Copy Citation Text

Haoran Shen, Puzheng Wang, Ming Lu, Chi Zhang, Jian Li, Qin Wang, "Tactile-assisted point cloud super-resolution," Chin. Opt. Lett. 23, 051102 (2025)

Download Citation

EndNote(RIS)BibTex Plain Text

Set citation alerts for article

Save article for my favorites

Paper Information

Category: Imaging Systems and Image Processing

Received: Jul. 5, 2024

Accepted: Nov. 14, 2024

Published Online: May. 14, 2025

The Author Email: Jian Li (jianli@njupt.edu.cn), Qin Wang (qinw@njupt.edu.cn)

DOI:10.3788/COL202523.051102

CSTR:32184.14.COL202523.051102

Topics

laser devices and laser physics

Lasers and Laser Optics

Laser physics

laser manufacturing

Instrumentation, Measurement and Metrology

Table 1. Feature Fusion Pipeline

Table 1. Feature Fusion Pipeline

Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Methoda

Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Methoda

Table 2. Quantitative Comparisons to Other Methods on the TSR-PDa

Table 2. Quantitative Comparisons to Other Methods on the TSR-PDa

Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Worka

Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Worka

Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)a

Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)a

Table 5. Training Speed and Inference Time for the Model With Different Numbers of Touches (r = 16)

Table 5. Training Speed and Inference Time for the Model With Different Numbers of Touches (r = 16)

Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Method^a

Table 1. Quantitative Comparisons for Different Numbers of Tactile Iterations by Our Method^a

Table 2. Quantitative Comparisons to Other Methods on the TSR-PD^a

Table 2. Quantitative Comparisons to Other Methods on the TSR-PD^a

Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Work^a

Table 3. Quantitative Comparisons Under Different Upsampling Rates Between the State-of-the-Art Work and Our Present Work^a

Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)^a

Table 4. Comparing the Upsampling Performance of Our Full Pipeline with Various Cases in the Ablation Study (r = 16)^a