An image stitching algorithm that combines enhanced optimal seam and optimized brightness is proposed to address the issue of ghosting and inconsistent brightness in panoramic images arising from large parallax and exposure differences. First, a deformation model based on minimizing projection biases was used for image registration to accurately align overlapping area. Second, an enhanced optimal seam algorithm was implemented between two intersections in the overlapping area to avoid information loss in the panoramic image. Finally, leveraging the Poisson fusion image, the energy functional of the ideal panoramic image gradient and a nonuniform illumination fitting model were constructed to optimize the brightness and improve the brightness consistency of the panoramic image. Experimental results show that compared with the algorithm proposed in reference [11], the proposed algorithm improves the structural similarity by 5.58% and peak-signal-to-noise ratio by 9.55% in terms of eliminating large parallax. Compared with before optimization, the proposed algorithm reduces the average gradient of the illumination component by 14.90% and improves the average gradient by 12.09% in terms of eliminating exposure differences. Thus, the algorithm can be used for image stitching in scenes with large disparities and exposure differences.