In order to investigate the wavelength selectivity of volume holographic gratings, applied as wavelength division multiplexing (WDM) filter, a 532 nm laser was used to record volume holographic gratings with finite size in doubly doped cubic lithium niobate crystal. Gratings of different size ratio were recorded in transmission configuration one by one and read in orthogonal scheme with a tunable infrared laser working in the optical communication band. The 3 dB bandwidth of wavelength selectivity curves corresponding to the grating size ratio of 4∶3， 4∶2 and 5∶2 were 0.48 nm， 0.45 nm and 0.43 nm respectively. The Bragg central wavelength of each grating is almost the same as theoretical prediction. This indicated that with larger gratings size ratio one can obtain narrower 3 dB bandwidth in the readout of volume holographic gratings. The wavelength selectivity of volume holographic WDM filter can be obviously improved by choosing a suitable grating size ratio and it is possible to meet the requirement of dense WDM.