To meet the real-time requirements of fabric defect detection in industry, a real-time fabric defect detection algorithm based on S-YOLOV3 (Slimming You Only Look Once Version 3) model is proposed. To develop this algorithm, the K-means clustering algorithm is used to determine the target prior frame for adapting to different sizes of defects. The YOLOV3 model is then pretrained to obtain the weight parameters, and the scaling factor γ is used in the batch normalization layer to evaluate the weight of each convolution kernel. The convolution kernel with weight value lower than the threshold is clipped to obtain the S-YOLOV3 model to achieve compression and acceleration. Finally, the pruned network is fine-tuned to improve the model detection accuracy. Experiment results reveal that the proposed model provides accurate detection of fabrics with different complex textures (average precision of 94%). After pruning, the detection speed is increased to 55 FPS. The obtained accuracy and real-time can meet the actual demand of industry.