A modal wavefront sensing technique of using binary holography optical element(BHOE),and a detail theoretical analysis on the principle of the modal wavefront sensor of this type are presented. The primary design approach of the BHOE is introduced. The effect on the unknown beam intensity of the aberration of single Zernike mode is also studied. The BHOE is encoded and designed with computer,and simulations are carried out to verify the feasibility and validity of the wavefront sensing approach. With the optical parallel processing ability,the speed of the wavefront sensor is highly improved,eliminating the computational overhead or post-processing in wavefront reconstruction stage. Compared with the conventional wavefront sensors,our technique is suitable for the large bandwidth situation,e.g. the atmosphere transmission in airborne enviroments,the requirements for high update rates,the large scintillation environments and so on.