One of the common faults in grate system is side plate offset, and when the side plate offset occurs, it will cause serious accidents and economic losses. Aiming at the problems of time-consuming, laborious, and low intelligent in the side plate offset detection method based on manual observation and inspection, a side plate offset detection method based on line segment detector (LSD) algorithm is proposed. The whole detection scheme including hardware equipment construction and data processing method is designed. The image information of the side plate of the grate trolley is collected by camera. According to the actual running condition of the grate trolley, the offset detection rules of the side plate of the grate trolley are set. Through region of interest capture and region division of the collected video frame images, LSD algorithm is used to detect the straight-line segment information in image, and the straight-line segment information of the trolley’s side plate obtained from the detection is processed by straight-line fusion and breakpoint difference calculation. According to the judgment rules, the offset fault of the trolley’s side plate is detected online and the alarm is given automatically. By testing the video image of trolley side plate collected in sintering plant of Baotou Steel, the program error detection rate is 1%, and the average detection and judgment time is less than 0.1 s, which can effectively replace the manual observation detection method to detect the offset fault of trolley side plate, and provide technical support for the offset detection of chain grate trolley side plate.