Infrared and Laser Engineering, Volume. 49, Issue 11, 20200269(2020)

Image sentiment classification via deep learning structure optimization

Jiachuan Sheng1,2, Yaqi Chen1, Jun Wang3, and Yahong Han4、*
Author Affiliations
  • 1School of Science and Technology, Tianjin University of Finance &Economics, Tianjin 300222, China
  • 2Laboratory of Fintech and Risk Management, Tianjin 300222, China
  • 3School of Management Science and Engineering, Tianjin University of Finance & Economics, Tianjin 300222, China
  • 4College of Intelligence and Computing, Tianjin University, Tianjin 300072, China
  • show less
    Figures & Tables(13)
    Overview of proposed algorithm
    Features visualization of the convolutional layers
    Channel visualization example of the fine-tuned model
    Neuron group visualization example of the fine-tuned model
    Class activation map of a positive class example image
    Class activation map of spliced images
    Visualization of spatial location features of a negative class image
    Visualization of neuron group features
    • Table 1.

      Details of auxiliary classifier

      辅助分类器详细信息

      View table
      View in Article

      Table 1.

      Details of auxiliary classifier

      辅助分类器详细信息

      Global average poolingOutput size of dropoutOutput size of full connectionOutput size of Softmax
      Kernel sizeOutput size
      Auxiliary classifier of Layer (4a)14×141×1×5121×1×5121×1×21×1×2
      Auxiliary classifier of Layer (4b)14×141×1×5121×1×5121×1×21×1×2
      Auxiliary classifier of Layer (4d)14×141×1×5281×1×5281×1×21×1×2
    • Table 2.

      Details of the Twitter dataset

      Twitter数据集详细信息

      View table
      View in Article

      Table 2.

      Details of the Twitter dataset

      Twitter数据集详细信息

      Five agreeFour agreeThree agree
      Positive581689769
      Negative301427500
      Total8821 1161 269
    • Table 3.

      Results of comparative experiment

      对比实验结果

      View table
      View in Article

      Table 3.

      Results of comparative experiment

      对比实验结果

      PCNN[25]SentiNet-A[3]Proposed algorithm
      注:加粗字体为每行最优值。
      Five agreeP0.7700.8950.900
      R0.8780.8780.922
      F1 0.8210.8860.911
      A0.7470.8510.881
      Four agreeP0.7330.8510.860
      R0.8450.8350.877
      F1 0.7850.8420.868
      A0.7140.8070.836
      Three agreeP0.7140.8230.836
      R0.8060.8090.848
      F1 0.7570.8140.842
      A0.6870.7770.807
    • Table 4.

      Comparative results of adding auxiliary classifiers at each layer

      各层添加辅助分类器作用对比

      View table
      View in Article

      Table 4.

      Comparative results of adding auxiliary classifiers at each layer

      各层添加辅助分类器作用对比

      NumberConvolution layer of auxiliary classifierAccuracy
      (1)No auxiliary classifier0.800
      (2)Layer (3a)0.817
      (3)Layer (3b)0.808
      (4)Layer (4c)0.798
      (5)Layer (4e)0.815
    • Table 5.

      Ablation experiment results

      消融实验结果对比

      View table
      View in Article

      Table 5.

      Ablation experiment results

      消融实验结果对比

      NumberAblation network structureAccuracy
      (1)Our algorithm0.881
      (2)Delete the auxiliary classifier of layer (4a)0.862
      (3)Delete the auxiliary classifier of Layer (4b)0.866
      (4)Delete the auxiliary classifier of Layer (4d)0.858
      (5)Delete the auxiliary classifier of Layer (4a) and Layer (4b)0.849
      (6)Delete the auxiliary classifier of Layer (4a) and Layer (4d)0.845
      (7)Delete the auxiliary classifier of Layer (4b) and Layer (4d)0.851
    Tools

    Get Citation

    Copy Citation Text

    Jiachuan Sheng, Yaqi Chen, Jun Wang, Yahong Han. Image sentiment classification via deep learning structure optimization[J]. Infrared and Laser Engineering, 2020, 49(11): 20200269

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Image processing

    Received: Jun. 11, 2020

    Accepted: Jul. 15, 2020

    Published Online: Jan. 4, 2021

    The Author Email: Han Yahong (yahong@tju.edu.cn)

    DOI:10.3788/IRLA20200269

    Topics