In this paper, Surface Enhanced Raman Spectroscopy (SERS) was used to realize the sensitive detection of 5-fluorouracil (5-FU). Silver nanoparticles (AgNPs) with different diameters were prepared by seed growth method. Crystal Violet (CV) was used as a probe molecule to evaluate the enhancement ability of the SERS substrate. The optimized AgNPs were used as SERS substrates and the standard solutions of 5-FU with different concentrations were detected by portable Raman spectroscopy. Finally, 5-FU spiked in human serum was determined after the pretreatment of serum. Protein was removed by organic precipitation to reduce detection interference and the detection limit of 5-FU in serum was 3.125 μg/mL.