In order to further improve the overall performance of the coupling cone structure optical fiber ultrasonic sensor and make it better serve the optical fiber ultrasonic nondestructive testing, the influence of the material parameters of the coupling cone on the response sensitivity of the sensor is simulated and analyzed by using the finite element analysis method in this paper. Based on this, 4 kinds of cone materials with excellent ultrasonic energy accumulation effect are selected and the best response cone angle is calculated to match with them. 4 optical fiber ultrasonic sensors based on 74° aluminum cone, 30° plexiglass cone, 130° polystyrene cone, and 126° natural rubber cone are optimized and designed. Experimental results show that these 4 sensors can effectively detect ultrasonic signals with a frequency of 1 MHz, and compared with the existing sensors, have a greater improvement in response sensitivity, which can effectively improve the sensor's sensing performance. Moreover, the structures are smaller and lighter, which can be used in more extensive occasions.