An acidic gas is an important basic chemical raw material used for synthesizing fertilizers, insecticides, explosives, dyes, and salts. Alternatively, inorganic acidic gases that leak into the air have harmful effects on the human health, infrastructure, and cultural relics. Therefore, the demand for inorganic acidic gas sensors for air quality monitoring and management has continuously increased, enabling the development of various sensing technologies. Among them, fiber-optic sensors are promising for acidic gas detection because of their excellent in-situ measurement, resistance to corrosion, anti-electromagnetic interference, long service life, and smart structure. In particular, fiber-optic sensors have proven to be very useful for the in-situ detection and distributed monitoring of multiple gas parameters. However, the sensitivity, selectivity, repeatability, and limits of detection of these sensors can be improved to achieve acceptable performance levels for practical applications. In this review, we introduce fiber-optic sensors based on structured optical fibers and fiber gratings for detecting H2 S, SO2, NO2, CO2, and N2O. The structures of the sensing regions, gas-sensitive materials, and measurement principles of these sensors are presented. The sensitivity, selectivity, limit of detection, and response time of the sensors are summarized. Finally, the future of fiber-optic sensors for the detection of inorganic acidic gases is discussed.