There are many successful applications of deep learning in solving challenging and difficult problems in recent years. An excellent example is its application in medical image analysis. Dr. Heng’s group is the first to employ 3D multi-level contextual 3D convolutional neural network framework for false positive reduction in automated pulmonary nodule detection. They further proposed a novel and efficient 3D CNN equipped with a 3D deep supervision mechanism to comprehensively address the challenges of optimization difficulties of 3D networks and inadequacy of medical training samples. Their successful deep learning applications cover a wide spectrum of medical image modalities， include histopathlogical imaging， ultrasound imaging， MR/CT imaging， dermoscopy imaging. Concurrently， there are also many significant and promising developments in virtual reality that are applicable for medical applications. Virtual reality based surgical simulation can provide a cost-effective and efficient way to train novices. In order to achieve the goal of delivering specialized training of a surgical procedure， one practical solution is to construct a realistic virtual environment through intelligent integration of medical imaging， motion tracking， physically based simulation， haptic feedback and visual rendering. In this review， Dr. Heng will present their recent works in using deep learning for medical image analysis and introduce some VR-based surgical simulators they have developed.