With the classical ensemble model, the nonsequential double ionization (NSDI) of Xe atoms is investigated by the elliptically polarized few-cycle laser pulses at the laser intensity of 2.5×1014 W/cm2 and the laser wavelength of 780 nm. The results show that the momentum spectra of the final-state correlated electron pairs from NSDI along the long axis of the polarization plane are mainly distributed in the first and third quadrants, showing strongly correlated behavior at the various carrier-envelop phases (CEPs). The rate of the total number for the final-state correlated electron pairs whose momentum spectra are distributed in the first and third quadrants shows decrease firstly then increase as CEP decreases. By analyzing the trajectories, the events of NSDI are due to the recollision at this laser pulse and the processes of NSDI are dominated by (e, 2e) ionization. In addition, the laser phase at the recollision also strongly depends on CEP.