In order to meet the increasing bandwidth demand of users in the future, starting from the key technology of physical layer of high-capacity optical access network, this paper summarizes the standardization status of the next generation passive optical network(NG-PON), reviews the advanced modulation formats in the physical layer, the enhanced power budget technology, the physical layer impairment and suppression based on the advanced digital signal processing(DSP) algorithms. In addition, the future reseach directions of NG-PON physical layer is prospected.