Opto-Electronic Engineering, Volume. 49, Issue 4, 210307(2022)

Satellite cloud image retrieval based on deep metric learning

Zhuzhang Jin... Xuyuan Fang, Yanhui Huang, Caoqian Yin and Wei Jin* |Show fewer author(s)
Author Affiliations
  • Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo, Zhejiang 315211, China
  • show less

    Overview: Meteorological satellites can monitor weather phenomena of different scales from the air, and the satellite cloud images obtained by them play an important role in weather analysis and forecast. In recent years, with the development of meteorological satellite technology, the spatial and spectral resolution of satellite cloud images and the acquisition frequency of imaging spectrometer have been continuously improved. How to manage massive satellite cloud images and design an efficient cloud image retrieval system has become a difficult problem for meteorologists. However, the traditional cloud image retrieval methods are difficult to obtain ideal retrieval accuracy and retrieval efficiency. Motivated by the impressive success of the modern deep neural network (DNN) in learning the optimization features of specific tasks in an end-to-end fashion, a cloud image retrieval method based on deep metric learning is proposed in this paper. Firstly, a residual 3D-2D convolutional neural network was designed to extract spatial and spectral features of cloud images. Since the features extracted by the traditional classify-based deep network may have greater differences intra-classes than inter-classes, the triplet strategy is used to train the network, and the cloud images are mapped into the metric space according to the similarity between cloud images, so that the distance of similar cloud images in the embedded space is smaller than that of non-similar cloud images. In model training, the convergence performance of traditional triplet loss is improved and the precision of cloud image retrieval is increased by adding a constraint on the distance between positive sample pairs to the lossless triplet loss function. Finally, through hash learning, the cloud features in the metric space are transformed into hash codes, so as to ensure the retrieval accuracy and improve the retrieval efficiency. Experimental results show that the mean average precision (mAP) of the proposed algorithm is 75.14% and 80.14% for the southeast coastal cloud image dataset and the northern hemisphere cloud image dataset respectively, which is superior to other comparison methods.Due to the traditional cloud image retrieval methods are difficult to obtain ideal retrieval accuracy and retrieval efficiency, a cloud image retrieval method based on deep metric learning is proposed. Firstly, a residual 3D-2D convolutional neural network is designed to extract spatial and spectral features of cloud images. Since the features extracted by the traditional classify-based deep network may have greater differences intra-classes than inter-classes, the triplet strategy is used to train the network, and the cloud images are mapped into the metric space according to the similarity between cloud images, so that the distance of similar cloud images in the embedded space is smaller than that of non-similar cloud images. In model training, the convergence performance of traditional triplet loss is improved and the precision of cloud image retrieval is increased by adding a constraint on the distance between positive sample pairs to the lossless triplet loss function. Finally, through hash learning, the cloud features in the metric space are transformed into hash codes, so as to ensure the retrieval accuracy and improve the retrieval efficiency. Experimental results show that the mean average precision (mAP) of the proposed algorithm is 75.14% and 80.14% for the southeast coastal cloud image dataset and the northern hemisphere cloud image dataset respectively, which is superior to other comparison methods.

    Tools

    Get Citation

    Copy Citation Text

    Zhuzhang Jin, Xuyuan Fang, Yanhui Huang, Caoqian Yin, Wei Jin. Satellite cloud image retrieval based on deep metric learning[J]. Opto-Electronic Engineering, 2022, 49(4): 210307

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category:

    Received: Sep. 23, 2021

    Accepted: --

    Published Online: May. 24, 2022

    The Author Email: Jin Wei (xyjw1969@126.com)

    DOI:10.12086/oee.2022.210307

    Topics