The long-tail problem is very common in recommendation system. It leads to recommending few and homogeneous products. We propose a new recommendation algorithm named GES4RL, which combines graph embedding with side information and reinforcement learning to solve long-tail problem. GES4RL is based on Node2Vec and negative feedback reinforcement learning. It constructs a weighted directed graph of product propagation and uses Node2Vec to learn the embedding of products. We used gated recurrent unit (GRU) to learn user's dynamic preferences and designed a negative feedback reinforcement learning model to generate the best recommendation strategy for long-tail products. Experimental results on User Behavior Dataset provided by TianChi show that the algorithm improves the diversity and hit rate of recommendations significantly.