To use the characteristics of pointer meter images without the limitation of existing reading recognition methods, a fully automatic reading recognition method based on a lightweight image semantic segmentation model is proposed. In the proposed method, the lightweight semantic segmentation network CGNet is modified by implementing the channel attention module SENet to enhance and aggregate image features and by deepening classification layers appropriately to predict more accurate semantic pixels of scale lines, pointers, and scale-range numbers. Then, according to the semantic segmentation results, an ellipse is fitted, and perspective transform between the ellipse and a standard circle is performed to correct skewed images. Scale lines and pointers are then extracted from the corrected images by postprocessing operations such as polar transform, image thinning, and vertical projection, and scale-range numbers are recognized using optical character recognition technology. Finally, the meter reading is calculated according to the scale range and relative position of the pointer and scale lines. An image dataset of pointer meters is constructed to validate the proposed method. Experimental results demonstrate that the proposed method realizes significant improvement of image semantic segmentation precision compared to existing lightweight models, and the average relative error of reading recognition for images on the test set is approximately 0.63%, which satisfies the requirements of practical applications.