Neuromorphic computing extends beyond sequential processing modalities and outperforms traditional von Neumann architectures in implementing more complicated tasks, e.g., pattern processing, image recognition, and decision making. It features parallel interconnected neural networks, high fault tolerance, robustness, autonomous learning capability, and ultralow energy dissipation. The algorithms of artificial neural network (ANN) have also been widely used because of their facile self-organization and self-learning capabilities, which mimic those of the human brain. To some extent, ANN reflects several basic functions of the human brain and can be efficiently integrated into neuromorphic devices to perform neuromorphic computations. This review highlights recent advances in neuromorphic devices assisted by machine learning algorithms. First, the basic structure of simple neuron models inspired by biological neurons and the information processing in simple neural networks are particularly discussed. Second, the fabrication and research progress of neuromorphic devices are presented regarding to materials and structures. Furthermore, the fabrication of neuromorphic devices, including stand-alone neuromorphic devices, neuromorphic device arrays, and integrated neuromorphic systems, is discussed and demonstrated with reference to some respective studies. The applications of neuromorphic devices assisted by machine learning algorithms in different fields are categorized and investigated. Finally, perspectives, suggestions, and potential solutions to the current challenges of neuromorphic devices are provided.