Laser & Optoelectronics Progress, Volume. 59, Issue 22, 2215009(2022)

Cross-Domain Spatial Co-Attention Network for Sketch-Based Image Retrieval

Lingzhi Yu* and Xifan Zhang
Author Affiliations
  • School of Electrical and Information Engineering, Tianjin University, Tianjin 300072, China
  • show less

    Sketch-based image retrieval uses hand-drawn sketches as input to retrieve corresponding natural images, allowing users to draw and find desired natural images when no accurate query images are available. Edge maps are commonly used as an intermediate modality to bridge the domain gap between sketches and natural images. However, existing methods ignore the inherent relationship between edge maps and natural images. Based on the assumption that natural images and their corresponding edge maps have similar key regions, this paper proposes a deep learning model based on a cross-domain spatial co-attention network. The proposed model derives the shared spatial attention mask from the fused feature of the edge map and natural image, and it combines the loss function and auxiliary classifier for end-to-end training. When compared with existing representative sketch-based image retrieval methods, the proposed method can effectively extract the features of sketches and natural images, with mean average precision (mAP) values of 0.933 and 0.799 on the Sketchy and TU-Berlin datasets, respectively, outperforming most representative methods.

    Tools

    Get Citation

    Copy Citation Text

    Lingzhi Yu, Xifan Zhang. Cross-Domain Spatial Co-Attention Network for Sketch-Based Image Retrieval[J]. Laser & Optoelectronics Progress, 2022, 59(22): 2215009

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Aug. 30, 2021

    Accepted: Oct. 27, 2021

    Published Online: Oct. 13, 2022

    The Author Email: Yu Lingzhi (yulingzhi777@sina.com)

    DOI:10.3788/LOP202259.2215009

    Topics