Optics and Precision Engineering, Volume. 29, Issue 12, 2944(2021)

Image caption of space science experiment based on multi-modal learning

Pei-zhuo LI... Xue WAN* and Sheng-yang LI |Show fewer author(s)
Author Affiliations
  • Key Laboratory of Space Utilization, Chinese Academy of Sciences, Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences, University of Chinese Academy of Sciences, Beijing100094, China
  • show less
    Figures & Tables(15)
    Framework of the algorithm
    Comparison of original U-Net and improved U-Net
    Bottom-up attention and CNN attention results
    Space science experiment picture categroy quantity chart
    Samples of space science experiment dataset
    Samples of semantic segmentation annotations of the dataset
    Growth experiment of Arabidopsis thaliana and rice in Tiangong-2
    Loss curves of Arabidopsis thaliana and rice growth experiment on Tiangong-2
    Comparisons between segmentation results of between Arabidopsis thaliana and rice growth experiment in Tiangong-2
    Vocabulary candidate extraction for Tiangong-2 Arabidopsis thaliana and rice growth experiment
    Comparison of image caption between Neuraltalk2 and this paper in space science experiment
    Comparison results of image caption between this paper and Neuraltalk2 in space droplet experiment
    • Table 1. Comparison between original U-Net and advanced U-Net

      View table
      View in Article

      Table 1. Comparison between original U-Net and advanced U-Net

      模型损失函数优化器上下采样次数激活函数网络输入
      原始U-Net带权值的交叉熵随机梯度下降4softmax572×572
      改进U-NetBCEDice损失函数RMSProp6sigmoid256×256
    • Table 2. Comparison table for evaluation of semantic segmentation algorithm in Tiangong-2 Arabidopsis thaliana and rice growth experiment

      View table
      View in Article

      Table 2. Comparison table for evaluation of semantic segmentation algorithm in Tiangong-2 Arabidopsis thaliana and rice growth experiment

      指标Mask R-CNN本算法
      水稻拟南芥水稻拟南芥
      FM0.187 40.523 40.927 20.966 4
      FO0.216 70.711
      FD0.279 1-0.186 5-0.091 3-0.028
      JM0.167 50.2020.663 90.882 2
      JO0.200.883 31
      JD0.227 6-0.105 7-0.243 6-0.075
    • Table 3. Evaluation results of this paper and Neuraltalk2

      View table
      View in Article

      Table 3. Evaluation results of this paper and Neuraltalk2

      ExperimentNeuraltalk2本算法
      METEORSPICEMETEORSPICE
      T20.04000.2720.230
      ZeroG-Flame0.0680.0030.0950.169
      Droplet-Ping-Pong0.1330.0710.110.138
      ISS-Flame0.110.0880.2280.319
    Tools

    Get Citation

    Copy Citation Text

    Pei-zhuo LI, Xue WAN, Sheng-yang LI. Image caption of space science experiment based on multi-modal learning[J]. Optics and Precision Engineering, 2021, 29(12): 2944

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Information Sciences

    Received: Apr. 29, 2021

    Accepted: --

    Published Online: Jan. 20, 2022

    The Author Email: WAN Xue (wanxue@csu.ac.cn)

    DOI:10.37188/OPE.2021.0244

    Topics