A self-starting all polarization-maintaining mode-locked figure-nine fiber laser was proposed, and different multi-soliton pulse output could be obtained by adjusting the pump power. When the pump power reached 224 mW, the laser could realize the self-starting of multi-pulse mode-locked state, and the pump power could be reduced to 47 mW under the mode-locked state. At the pump power of 143 mW, 101 mW, 89 mW, 76 mW, 65 mW and 55 mW, the output phenomena of loosely bound state pulses, 2~5 multi-soliton pulses and stable single pulse were observed, respectively. The obtained soliton pulse repetition rate is 8.68 MHz. In the loosely bound state pulses, the signal-to-noise ratio (SNR) is as high as 70.4 dB, the main pulse width is 5.78 ps, and the spacing between two adjacent pulses is 50.04 ps. In the single-pulse state, the SNR is 65.8 dB, the pulse width is 6.17 ps, the average output power is 0.305 mW, and the single pulse energy is 0.035 nJ. In addition, the output power of the laser within 10 h was measured, demonstrating that the structure was highly stable. The experimental results are helpful for further understanding of multi-soliton dynamics, and the laser proposed has the advantages of simple structure, self-starting, good stability, and switchable mode-locked pulse, which is valuable for promoting the application of fiber lasers.