Journal of Applied Optics, Volume. 41, Issue 2, 337(2020)

Using TensorRT for deep learning and inference applications

Lijun ZHOU1... Yu LIU1, Lu BAI2, Fei LIU1 and Yawei WANG1 |Show fewer author(s)
Author Affiliations
  • 1Xi’an Institute of Applied Optics, Xi’an 710065, China
  • 2Xi’an North Electro-optic Science & Technology CO.LTD., Xi’an 710043, China
  • show less

    TensorRT is a high-performance deep learning and inference platform. It includes a deep learning and inference optimizer as well as runtime that provides low latency and high throughput for deep learning and inference applications. An example of using TensorRT to quickly build computational pipelines to implement a typical application for performing intelligent video analysis with TensorRT was presented. This example demonstrated four concurrent video streams that used an on-chip decoder for decoding, on-chip scalar for video scaling, and GPU computing. For simplicity of presentation, only one channel used NVIDIA TensorRT to perform object identification and generate bounding boxes around the identified objects. This example also used video converter functions for various format conversions, EGLImage to demonstrate buffer sharing and image display. Finally, the GPU card V100 was used to test the TensorRT acceleration performance of ResNet network. The results show that TensorRT can improve the throughput by about 15 times.

    Tools

    Get Citation

    Copy Citation Text

    Lijun ZHOU, Yu LIU, Lu BAI, Fei LIU, Yawei WANG. Using TensorRT for deep learning and inference applications[J]. Journal of Applied Optics, 2020, 41(2): 337

    Download Citation

    EndNote(RIS)BibTexPlain Text
    Save article for my favorites
    Paper Information

    Category: OE INFORMATION ACQUISITION AND PROCESSING

    Received: Jun. 17, 2019

    Accepted: --

    Published Online: Apr. 23, 2020

    The Author Email:

    DOI:10.5768/JAO202041.0202007

    Topics