Terahertz holographic reconstructed images are prone to boundary blur. Therefore, this study proposes a segmentation method based on optimized region growth by evolutionary algorithms. First, the proposed method is used to perform bilateral filtering and morphological erosion on the original images to obtain the seeds of the region growth. Second, genetic algorithm and differential evolution algorithm are used to perform threshold optimization to limit the region growth. Subsequently, the segmentation results of the terahertz holographic images are obtained. Average structure similarity (MSSIM) is used as an objective evaluation for assessing the algorithm''s effectiveness. Segmentation results show that the region-growing algorithm optimized by the evolutionary algorithm has a good segmentation effect. Moreover, the MSSIM can reach 0.8 or higher. Finally, to compare the optimization performance of two evolutionary algorithms, the algorithms are applied to visible light images. According to the segmentation results of the images, it is concluded that the differential evolution algorithm is superior to the genetic algorithm in terms of speed and searchability.