Data-driven Machine Learning (ML) has been widely used in materials performance optimization and novel materials design due to its ability to quickly fit potential data patterns and achieve accurate prediction. However, the results of data-driven ML are often inconsistent with the materials basic theory or principle, which results mainly from the lack of the guidance of materials domain knowledge, e.g., the correlation among descriptors and the driving mechanism associated with the properties. Herein, by analyzing the characteristics of materials data and the modeling principle of data-driven ML methods, we clarify the three main contradictions occurring to the application of ML in materials science, i.e., the contradictions between high dimension and small sample, complexity and accuracy of models, learning results and domain knowledge. Following this, we propose the ML method embedded with materials domain knowledge to reconcile these three contradictions. Further, surrounding the whole ML process including target definition, data collection and preprocessing, feature engineering, model construction and application, we explore some key techniques to realize domain knowledge embedding by summarizing the related basic and exploratory efforts. Finally, opportunities and challenges facing the ML method embedded with domain knowledge are also discussed.