Innovations in artificial intelligence, particularly artificial neural networks, have revolutionized applications in many areas, such as big-data search, computer recognition, and language and image recognition. The development of nanophotonics in the past decades has brought physical perspectives and different approaches to the implementation and the development of traditional artificial neural network technologies, especially optical neural networks. On the one hand, nanophotonics is a field studying the interaction of light and matter at the nanoscale, which can lead to new techniques, such as super-resolution optical lithography and super-resolution optical imaging technology, therefore in turn promoting the implementation of optical neural networks with multiple functions at the micro/nano scale. On the other hand, due to the characteristics of multi-bands, high speed, and low power consumption of light propagation, nanophotonics is accelerating the development of optical neural networks with compact size, high density, and low power consumption. Meanwhile, the development of artificial neural networks has also promoted neural network algorithms (such as reverse design and deep learning) as a new toolbox for the design of novel nanophotonics devices to meet the growing requirements of the function, volume, integration, and computing function of nano-photonic devices. In this paper, starting with the development of neural networks,we review the development of artificial neural networks, especially the development of optical neural networks. The reciprocal development between artificial neural networks and nanophotonics is reviewed.