Laser & Optoelectronics Progress, Volume. 61, Issue 22, 2215005(2024)

Fatigue Driving Detection Under Low Illumination Using Image Enhancement and Facial State Recognition

Yang Zhao1,2, Jialong Miao1、*, Xuefeng Liu1, Jincheng Zhao3, and Sen Xu1,2
Author Affiliations
  • 1The College of Computer Science and Technology, Shenyang University of Chemical Technology, Shenyang 110142, Liaoning , China
  • 2Key Laboratory of Intelligent Technology for Chemical Process Industry of Liaoning Province, Shenyang 110142, Liaoning , China
  • 3The College of Information Engineering, Shenyang University of Chemical Technology, Shenyang 110142, Liaoning , China
  • show less
    Figures & Tables(27)
    YOLOv5s model structure
    The visualization results of each image enhancement method. (a) Original image; (b) LIME; (c) EnlightenGAN; (d) SCI
    StemBlock module
    ShuffleNetV2 module. (a) Basic module S_block1; (b) subsampling module S_block2
    Bottleneck module and inverted bottleneck module. (a) Bottleneck module; (b) inverted bottleneck module
    The structure of CBAM
    CIB module
    CIBC3 module
    Loss function image with different parameters
    Improved YOLOv5s model
    Result instance
    Schematic diagram of eye area
    Schematic diagrams of an image capture of the eye and mouth. (a) Cropped image of the eye region; (b) cropped image of the mouth region
    The structure of FSR-Net
    Fatigue driving detection model
    Examples of dateset
    Examples of fatigue driving dataset. (a) Examples of the YawDD public dataset; (b) examples of the self-built dataset
    Examples of image enhanced self-built dataset
    Examples of fatigue detection results. (a)‒(d) YawDD public dataset detection results; (e)‒(h) self-built dataset detection results
    • Table 1. Comparative test of each image enhancement method

      View table

      Table 1. Comparative test of each image enhancement method

      Enhancement methodmAP@0.5 /%
      71.36
      LIME70.17
      EnlightenGAN70.33
      SCI72.19
    • Table 2. Experimental results on the DARK FACE dataset

      View table

      Table 2. Experimental results on the DARK FACE dataset

      ModelmAP@0.5 /%Parmas /106FLOPs /109
      SSD57.3426.29242.71
      Faster RCNN62.52137.18398.12
      YOLOv5s69.817.2616.78
      YOLOv771.6737.21103.58
      YOLOv869.2811.1927.96
      Proposed model72.192.756.14
    • Table 3. Ablation experiment

      View table

      Table 3. Ablation experiment

      No.ShuffleNetV2StemBlockIBC3CIBC3SCImAP@0.5 /%Parmas /106FLOPs /109
      1×××××69.817.2616.78
      2××××68.932.927.47
      3×××69.752.916.31
      4××70.492.736.00
      5×71.362.756.14
      672.192.756.14
    • Table 4. Information of eye dataset

      View table

      Table 4. Information of eye dataset

      DatasetTraining setValidation setTest setTotal
      CEW19392422422423
      YawDD28803603603600
      Selt-built dataset28803603603600
    • Table 5. Information of mouth dataset

      View table

      Table 5. Information of mouth dataset

      DatasetTraining setValidation setTest setTotal
      YawDD28803603603600
      Selt-built dataset28803603603600
    • Table 6. FSR-Net detection accuracy

      View table

      Table 6. FSR-Net detection accuracy

      DatasetAccuracy ofeye /%Accuracy of mouth /%
      Average accuracy /%97.89
      CEW97.22
      YawDD97.5098.06
      Selt-built dataset98.0698.61
    • Table 7. Comparison of detection results between FSR-Net and different networks

      View table

      Table 7. Comparison of detection results between FSR-Net and different networks

      AlgorithmAverage accuracy /%Params /106
      MobileNetV295.533.50
      ShuffleNetV295.711.36
      GhostNet96.497.30
      ResNet5090.6425.64
      EMs-Net97.414.84
      EMSD-Net97.520.84
      Proposed algorithm97.890.45
    • Table 8. Experimental results of different models

      View table

      Table 8. Experimental results of different models

      ModelYawDD /%Selt-built dataset /%Average time expenditure /ms
      MTCNN+EMs-Net93.2681.5019
      DWC-based one-stage algorithm94.3879.0024
      RetinaFace+EMSD-Net95.5083.5040
      Proposed model96.0794.5027
    Tools

    Get Citation

    Copy Citation Text

    Yang Zhao, Jialong Miao, Xuefeng Liu, Jincheng Zhao, Sen Xu. Fatigue Driving Detection Under Low Illumination Using Image Enhancement and Facial State Recognition[J]. Laser & Optoelectronics Progress, 2024, 61(22): 2215005

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: Machine Vision

    Received: Feb. 20, 2024

    Accepted: Mar. 25, 2024

    Published Online: Nov. 19, 2024

    The Author Email: Jialong Miao (igxiaodingdang@163.com)

    DOI:10.3788/LOP240711

    CSTR:32186.14.LOP240711

    Topics