A novel two-stage correlation double sampling (CDS) circuit is proposed to remove the fixed pattern noise (FPN) for a compact time of flight (TOF) array detector with the time-amplitude converter (TAC) structure. Each CDS stage only adopted two switch transistors and one sampling capacitor, which was much simpler than the traditional fully differential ones. Fabricated in SMIC 0.18 μm standard CMOS technology, the CDS circuit merely occupied a small area of 40 μm×35 μm and consumed the low static power of 301 μW. Experimental results show that the proposed two-stage CDS circuit has a high linearity of 99.98 % and a wide output swing of 0.9 V under 1.8 V supply. The total FPN is reduced by more than 54%. The proposed CDS scheme can effectively eliminate the FPN from the pixels and column lines on the array, which is very suitable for high-density TAC-based TOF detectors.