Based on light field reconstruction and motion recognition technique, a penetrable interactive floating 3D display system is proposed. The system consists of a high-frame-rate projector, a flat directional diffusing screen, a high-speed data transmission module, and a Kinect somatosensory device. The floating occlusion-correct 3D image could rotate around some axis at different speeds according to user’s hand motion. Eight motion directions and speed are detected accurately, and the prototype system operates efficiently with a recognition accuracy of 90% on average.