Laser & Optoelectronics Progress, Volume. 62, Issue 9, 0922002(2025)

Design of a Split Optical See-Through Head-Mounted Display System with Integrated Multispectral Sensors

Jiaxi Duan1,2、**, Dewen Cheng1,2、*, Lianbing Wang2, Shuxian Sun2, Qichao Hou2,3, Yang Wang2, Pengli Wang2, Xin Che2, Weihong Hou2,3, and Donghua Wang3
Author Affiliations
  • 1Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing 100081, China
  • 2Ned Co., Ltd., Beijing 100081, China
  • 3Dega Smart Photoelectric Technology Co., Ltd.,Zhenjiang 212132, Jiangsu , China
  • show less
    Figures & Tables(20)
    Functional architecture diagram of OST-HMD system with integrated multi-spectral sensors
    Schematic diagram of split OST-HMD
    Schematic diagram of signal transmission architecture for head-mounted display module
    Frameflow of integrated multi-spectral sensors signal acquisition system
    Layout of OST display module system
    Components and prototype of the optical display module. (a) All components of the display system; (b) overall appearance of the display module
    Physical picture of the low-light camera module
    Physical picture of the infrared light camera module
    Physical picture of the visible light camera module
    Self-developed driver board circuit diagram
    Binocular display driver board. (a) Front layout of the driver board; (b) back layout of the driver board
    Handheld prototype structure of integrated multispectral signal acquisition module and freeform prism display module
    The prototype integrating multi-spectral signal acquisition module and display module. (a) Whole structure prototype with handheld structure and movable bracket; (b) head-mounted structure integrating multi-spectral signal acquisition module and display module
    Embedded core computing platform
    The display effect of the optical display module. (a) Input image displayed in the MicroOLED; (b) output image taken by the camera at the system output pupil
    Fusion of a virtual cup with a real cup. (a) Real cup; (b) fusion display of virtual cup and real cup
    Images captured by different sensors. (a) Image captured by low-light module; (b) image captured by infrared module; (c) image captured by visible light module
    • Table 1. RK3399 chip parameter table

      View table

      Table 1. RK3399 chip parameter table

      ParameterExplanation
      CPU• Architecture: Dual Cortex-A72 + Quad Cortex-A53, 64-bit CPU
      • Frequency is up to 1.8 GHz
      GPU• Mali-T860 GPU, OpenGL ES1.1/2.0/3.0/3.1, OpenCL
      • Supports AFBC (ARM frame buffer compression)
      Memory• Dual channel DDR3-1866/DDR3L-1866/LPDDR3-1866/LPDDR4
      • Support eMMC 5.1 with HS400, SDIO 3.0 with HS200
      Multi-media• 4K VP9 and 4K 10 bit H265/H264 video decoders, up to 60 frame/s
      • 1080P other video decoders (VC-1, MPEG-1/2/4, VP8)
      • 1080P video encoders for H.264 and VP8
      • Video post processor: de-interlace, de-noise, enhancement for edge/detail/color
      Display• Dual VOP: one supports 4096×2160 with AFBC supported; the other supports 2560×1600
      • Dual channel MIPI-DSI (4 lanes per channel)
      • eDP 1.3 (4 lanes with 10.8 Gb/s) to support display, with PSR
      • HDMI 2.0a for 4K 60 Hz with HDCP 1.4/2.2
      • DisplayPort 1.2 (4 lanes, up to 4K 60 Hz)
      • Supports Rec.2020 and conversion to Rec.709
      Interface• Dual 13M ISP and dual channel MIPI CSI-2 receive interface
      • Dual USB 3.0 with type-C supported
      • PCIe 2.1 (4 full-duplex lanes )
      • Embedded low power MCU for other applications
      • 8 channels I2S supports 8 channels RX or 8 channels TX
      Package• FCBGA828 21mm×21mm, 0.65mm pitch
    • Table 2. Low-light camera module specification sheet

      View table

      Table 2. Low-light camera module specification sheet

      DatasetParameter
      DetectorHigh sensitivity camera core
      Detector size1 inch
      Resolution1024×768
      Frame rate25 Hz
      Pixel size13 μm
      Weight(include lens)35.13 g
      Length size≤38 mm
    • Table 3. Infrared light camera module specification sheet

      View table

      Table 3. Infrared light camera module specification sheet

      DatasetParameter
      DetectorUncooled micro thermal module
      Resolution640×512, 384×288
      Pixel size12 μm
      Frame rate25 Hz, 50 Hz
      Weight(include lens)≤26.4 g
      Length size≤37 mm
    Tools

    Get Citation

    Copy Citation Text

    Jiaxi Duan, Dewen Cheng, Lianbing Wang, Shuxian Sun, Qichao Hou, Yang Wang, Pengli Wang, Xin Che, Weihong Hou, Donghua Wang. Design of a Split Optical See-Through Head-Mounted Display System with Integrated Multispectral Sensors[J]. Laser & Optoelectronics Progress, 2025, 62(9): 0922002

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Optical Design and Fabrication

    Received: Jul. 9, 2024

    Accepted: Sep. 9, 2024

    Published Online: Apr. 23, 2025

    The Author Email: Jiaxi Duan (3220195071@bit.edu.cn), Dewen Cheng (cdwlxk@bit.edu.cn)

    DOI:10.3788/LOP241652

    CSTR:32186.14.LOP241652

    Topics