A scheme of information hiding for H.264/AVC based on encoding mode is proposed. Secret information is embedded in I-frame, P-frame and B-frame through adjustment of the encoding mode of given macroblocks. For the macroblocks provided with the encoding mode of intra 4×4, secret data is embedded by modulating the encoding mode of some 4×4 block. For other macroblocks in P-frame and B-frame, the hiding of information is implemented by modifying the encoding mode of macroblock. After the modulation of mode, an optimized processing for the macroblock is carried out to improve the coding efficiency. By introducing the rate-distortion cost during the process of mode modulating, the proposed algorithm achieves a better balance of rate-distortion. Meanwhile, it has less influence on the quality of video and the bit-rate of video. This scheme could detect the secret information rapidly, which meets the requirement of the real-time processing of video. Experimental results also demonstrate that it is an extremely effective scheme.