In order to improve the effect and stability of stereo matching algorithm, we propose a guided filtering algorithm based on adaptive aggregation region in Hue-Saturation-Value (HSV) color space. We calculate initial matching cost by the interaction of color and transverse gradient in combination with the structure and texture information of the image. Then, we calculate the length of the adaptive support arm of each point based on the color and distance information in HSV color space, which solves the problem that the change trend of the red, green and blue colors in the picture is similar and cannot effectively reflect the picture information. We construct the adaptive aggregation region using a transverse arm at each point on the longitudinal arm of the center point, and aggregate the cost space in the adaptive aggregation region by guided filtering. In order to avoid the problem that the support window is too small due to the fluctuation of the neighborhood information, we set the minimum range of the arm length. In the process of post-processing, we use the left and right consistency detection and the peak ratio detection method to find mismatching points, and correct the disparity map by the nearest neighbor matching and weighted median filtering. The experiments are carried out on standard images of Middlebury platform. Results show that average matching error of the proposed algorithm is 5.24%, and the matching error is reduced by 0.92% compared with the that of pre-improvement adaptive window algorithm. The proposed algorithm has better edge preservation effect and the robustness of the algorithm parameters is good.