Yb3+ doped crystal material, an important laser material, has promising applications in the fields of ultra short pulse laser and high power laser. However, it was very difficult to determine the crystal field parameters of Yb3+, especially of the hosts with low- symmetry. Ab-initio calculation was an important way to solve this problem. In this paper, the ab-initio DV-Xα method and the effective Hamiltonian model were introduced, which was suitable for calculating rare earth ion doped in hosts. With the ab-initio DV-Xα method and effective Hamiltonian model, the crystal field parameters of Yb3+ doped in the tantalite and niobate with M-type and M'-type were calculated. It is concluded that the part energy level structure fits well with the experimental spectra. And the crystal field parameters of Yb3+ doped in the tantalite and niobate show regular changes with the atomic numbers of rare earth. It is shown that DV-Xα method and effective Hamiltonian model are effective methods to calculate the crystal field of Yb3+ doped in the tantalite and niobate with low- symmetry, Yb3+ doped in the tantalite and niobate is the potential laser medium for high eifficiency laser operation.