A T-shaped cavity coupled metal-insulator-metal (MIM) waveguide structure with a rectangle cavity is proposed based on the transmission characteristics of surface plasmon polaritons (SPPs) in a sub-wavelength structure. The Fano resonance occurs due to the destructive interference between the narrow discrete state formed by the T-shaped cavity and the broad continuous state formed by the rectangular cavity. The coupled mode theory (CMT) is adopted to analyze the formation mechanism of Fano resonance. The finite element method is used to simulate this structure and analyze the influences of structural parameters on its refractive index sensing characteristics. The results show that after the optimization of structural parameters, its figure of merit is 6.04×104, and the sensitivity is 1120 nm/RIU. This research can provide a theoretical reference for the integration of photonic circuits and the design of nanoscale sensors in the future.