An all-fiber Mamyshev oscillator with a single amplification arm is experimentally demonstrated to achieve high-energy and high-average-power ultrafast pulse output, with the initiating of an external seed pulse. In the high-energy operation, a maximum single-pulse energy of 153 nJ is achieved at a repetition rate of 9.77 MHz. After compression with a pair of diffraction gratings, a measured pulse width of 73 fs with a record energy of 122.1 nJ and a peak power of 1.7 MW is obtained. In the high-average-power operation, up to 5th harmonic mode locking of the oscillator is realized via slightly adjusting the output coupling ratio and the cavity length. The achieved maximum output power is 3.4 W at a repetition rate of 44.08 MHz, while the corresponding pulse width is compressed to around ~100 fs. Meanwhile, the system is verified to be operated reliability in both high-energy and -average-power operation regimes through assessing its short- and long-term stabilities. To the best of our knowledge, these are the highest records in pulse energy and average power delivered from a single all-fiber ultrafast laser oscillator with picosecond/femtosecond pulse duration. It is believed that even higher-energy and -average-power ultrafast laser can be realized with the proposed laser scheme through further increasing the core diameter of the all-fiber cavity, providing promising sources for advanced fabrication, biomedical imaging, laser micromachining, and other practical applications, as well as an unprecedented platform for exploring undiscovered nonlinear dynamics.