In order to improve the mechanical properties of recycled aggregate concrete (RAC), ramie fiber (RF) was added into the concrete. The effects of recycled coarse aggregate (RCA) and RF on compressive strength, flexural strength and splitting strength of concrete were studied. The effects of RF on concrete properties and the improvement mechanism were further explored through nitrogen adsorption method and scanning electron microscope analysis. The mechanical properties of recycled aggregate concrete firstly increase and then decrease with the increase of RF content. When the replacement rate of RCA is 30% (mass fraction), the bending strength and splitting strength of the concrete with 0.5% (volume fraction) RF is 11.5% higher than that without RF. It can be seen from the microscopic analysis that the high water absorption rate of RF can make the cement paste on the fiber surface produce hydration reaction, and the generated hydrated calcium silicate gel fills the pores and thus increases the compressive strength of concrete. However, RF is prone to agglomerate when mixing concrete, forming weak areas, increasing internal defects, and inhibiting the improvement of compressive strength. The addition of RF has a relatively small effect on the improvement of compressive strength of concrete. RF surface is rough and has vertical stripes, which increases the bond strength of both sides of concrete cracks, acts as a bridge on both sides of concrete cracks, limits the development of concrete cracks under tensile load, and can greatly improve the flexural and splitting strength of concrete.