As Pattern of Oriented Edge Magnitude (POEM) method can not acquire enough feature description information in illumination condition changes drastically, this paper analyzes the characteristic of relative gradient magnitude images and proposes a Relative Gradient Histogram Feature(RGHF) description method. The method decomposes the relative gradient magnitude image into several sub images according to the orientations of gradient. Each of these sub images is then filtered and encoded by using Local Binary Patterns(LBPs). Finally, all the encoded LBP histogram features are connected by a lexicographic ordering and are reduced to a low-dimensional subspace to form the RGHF, which is an illumination robust low-dimensional histogram feature. Experimental results on FERET and YaleB subsets indicate when the illumination variation is relative small, the recognition performance of the RGHF is comparable with that of the POEM, superior to that of the LBP significantly. Moreover, when the illumination variation is drastic, the recognition performance of RGHF is at least 5% higher than that of the POEM, more better than those of the POEM and LBP.