In order to ensure the reliability and stability of optical fiber communication system, an in-band optical signal-to-noise ratio(OSNR) monitoring technique is proposed, which is independent of modulation format and insensitive to dispersion and polarization. By adjusting the center wavelength of the broadband tunable optical bandpass filter, the optical power of the channel on the link is sampled, and the measured optical power is used as the input feature of the machine learning algorithm Gaussian process regression(GPR) to accurately estimate and monitor the OSNR of the channel on the link. The experimental results show that in the OSNR range of 2-30 dB, the technology can achieve accurate OSNR monitoring, and is robust to chromatic dispersion(CD), polarization mode dispersion(PMD) and cascaded filtering effect (CFE), which is independent of modulation format and rate.