In advanced autonomous driving, the fusion of infrared camera and LiDAR can realize the vehicle's all-day, all-weather accurate perception of the surrounding complex environment. Accurate alignment is an indispensable step to realize multisensor fusion, and it is necessary to realize accurate alignment before fusion. Because infrared cameras have no color information, the traditional calibration method can be used for infrared camera and the LiDAR, and the calibration can only be performed based on a special heating target. However, such methods are costly, cumbersome to operate, and have limited calibration distances and low accuracy. Therefore, this study proposes an infrared and LiDAR calibration algorithm for road scenes. The proposed algorithm does not need a special target and can achieve accurate calibration at different distances according to the characteristics of the vehicles, pedestrians, lane lines, and road poles in different directions. Experimental results show that the proposed scheme can improve the robustness and security of automatic driving systems in harsh environments such as night, rain, fog, sand, and backlight. The proposed scheme can achieve more than 90% calibration accuracy for pedestrians and cars within longitudinal distances of 0 to 100 m and lateral distances of -10 m to 10 m.