A framework for human action recognition by learning pose dictionary based on human contour representation is proposed. A new pose feature based on Procrustes analysis and local preserving projection is proposed, which can extract shape information from human motion video which is invariant to translation, scaling and rotation. Moreover, it can extract discriminative subspace information when preserving local manifold structure of human pose. After the pose feature is extracted, a human action recognition framework based on pose dictionary learning is proposed. Class-specific dictionaries are trained individually on all training frames of each class to build the whole pose dictionary by concatenating all class-specific dictionaries. The test video is classified with the minimum reconstruction error on the learned dictionary. Experimental results on Weizmann and MuHAVi-MAS14 dataset demonstrate proposed method outperforms most classical methods. Especially, classification rate of this method on MuHAVi-MAS14 dataset achieves a considerable boost compared with that of state-of-the-art approaches.