The hydrophobic corrosion inhibitor makes the concrete have an excellent hydrophobic effect and inhibits the medium transmission in the corrosive environment. However, the mechanism of action is not clear. In this paper, the hydration behavior of cement paste with different content of carboxylic acid type hydrophobic agent (CAHA) was studied, and its effects on the hydration heat, hydrate phase, surface contact angle, and compressive strength of cement were investigated. The results show that CAHA reduces the first hydration exothermic peak of cement paste and delays the hydration of cement. In the early stage of hydration, CAHA inhibits the formation of ettringite and calcium hydroxide, which is more significant at high dosage. CAHA reduces the early compressive strength of mortar to a certain extent and has no obvious effect on 28 d compressive strength. CAHA significantly reduces the water absorption of mortar and makes it hydrophobic. Through comparison, the optimum dosage of CAHA is recommended to be 6% of the mass of cementitious material, at which time the loss of compressive strength of the cement mortar is smaller and the erosion resistance is significantly improved.