Opto-Electronic Science, Volume. 3, Issue 9, 230033(2024)

Edge enhanced depth perception with binocular meta-lens

Xiaoyuan Liu... Jingcheng Zhang, Borui Leng, Yin Zhou, Jialuo Cheng, Takeshi Yamaguchi, Takuo Tanaka* and Mu Ku Chen** |Show fewer author(s)
Figures & Tables(5)
Schematic of the edge-enhanced spatial computing with binocular meta-lens. There are two letter objects in front of the binocular meta-lens, which are texture-less and have no background. A binocular meta-lens is designed and fabricated to develop the stereo vision system for texture-less spatial computing scenarios. An edge-enhanced depth perception is realized with the support of a proposed neural network.
Optical and SEM images of fabricated binocular meta-lens. (a) Optical image of the binocular meta-lens. (b) The zoomed-in top-view SEM image of the meta-lens. (c) The zoomed-in tilted-view SEM image at the edge of the meta-lens.
Disparity computation with neural network. (a) Architecture overview of proposed neural network H-Net with H-Module. The stereo images are processed by weight-sharing backbones to extract features. These features are then combined using cross-pixel interaction and cross-view interaction in an H-Module. A 4D cost volume is created from the left and right image features, which is then used in a 3D CNN for depth estimation. A disparity regression module is performed before the final disparity map prediction. (b) Detailed pipelines of the cross-pixel interaction. The left and right feature maps are flattened and processed through separate fully connected layers to generate Query, Key, and Value vectors. The inner product is utilized to compute the similarity between Query and Key, resulting in weight coefficients for each Key. These coefficients are used for cross-pixel attention, associating each Key with its corresponding Value. The weighted Values are aggregated to produce enhanced features. (c) Detailed pipelines of the cross-view interaction. The difference from the cross-pixel interaction is the inner product of Key and Query vector comes from different stereo views.
Characterization of binocular meta-lens. (a) X-Z plane focusing profiles of left and right meta-lens under 532 nm of wavelength. The measured focal lengths of left and right meta-lenses are 10.048 mm and 10.046 mm, respectively, which are denoted by yellow dashed lines. (b) Designed phase distribution of the meta-lens. (c) Corresponding measured phase distribution of the meta-lens in (b).
Edge-enhanced depth perception of various objects. The first column is the raw left image. The second column is the corresponding depth map. The third column is the edge-enhanced depth map. The second and third columns use the same color bar on the right of the third column. The fourth column is the integration image of the raw image and edge-enhanced depth map. (a) Two pieces of transparent plastic paper printed with "RIKEN" and "CITYU" in black letters are placed at 16.0 cm and 12.8 cm, respectively. (b) A piece of sketch paper printed with a tilted three-dimensional building is located at 17.3 cm as the background. The front ends of the two toy cars are approximately 12.9 cm and 15.7 cm, respectively. (c) The two architectural sketches are at 13.5 cm and 16.5 cm, respectively. (d) The background architecture sketch is positioned at 17.3 cm. The depth of a toy car's body ranges from 12.5 cm to 15.5 cm.
Tools

Get Citation

Copy Citation Text

Xiaoyuan Liu, Jingcheng Zhang, Borui Leng, Yin Zhou, Jialuo Cheng, Takeshi Yamaguchi, Takuo Tanaka, Mu Ku Chen. Edge enhanced depth perception with binocular meta-lens[J]. Opto-Electronic Science, 2024, 3(9): 230033

Download Citation

EndNote(RIS)BibTexPlain Text
Save article for my favorites
Paper Information

Category: Research Articles

Received: Sep. 26, 2023

Accepted: Dec. 18, 2023

Published Online: Dec. 13, 2024

The Author Email: Tanaka Takuo (TTanaka), Chen Mu Ku (MKChen)

DOI:10.29026/oes.2024.230033

Topics