To automate present Automatic Optical Inspection (AOI) systems, an intelligent method based on incremental clustering for solder joint inspection is proposed in this paper. Firstly, to meet the demands of practical production, the framework for an intelligent AOI system is designed. Then, all the defects of solder joints are classified into several different types according to their appearances, and the color features in critical regions are extracted. The samples in each class are clustered into several subclasses so that the system is able to inspect solders from different batches. Finally, a new incremental clustering algorithm is proposed. The AOI system can automatically adjust inspection parameters according to the feedback from the repair station. To improve training efficiency, only a few samples are selected. The method proposed is used in an AOI to inspect solder joints, and the inspecting accuracy can reach 96.5% while each solder inspection takes 9.3 ms. The experimental result demonstrates that the proposed method can detect accurately a solder defect from different patches, and can be modified for different manufacturing processes. The intelligence level of the system using the proposed method is high, and it can be used in practical application.