Compared with glass microcavity, a calcium fluoride (CaF₂) crystalline microcavity has the advantages of small absorption coefficient, fewer defects, higher purity, and being insensitive to ambient humidity. It has potential applications in microwave photonics, gyroscopes, and nonlinear optics. In this paper, a spheroid CaF₂ millimeter crystalline microcavity is fabricated by the ultra-precision machining technology. To fabricate such a microcavity, a home-made precision machining system is developed. The shape of the fabricated microcavity is a spheroid, and the surface roughness of the edge of the microcavity is as low as 1.97 nm. Efficient coupling between the tapered fiber waveguide and the CaF₂ crystalline microcavity is achieved. The coupling system exhibits an ultra-high-quality (Q) factor up to ~10 ⁸ and a free-spectral-range as low as ~0.03 nm. These results demonstrate that the fabrication method for the CaF₂ crystalline microcavity is significant, and can promote its applications. The characteristics of CaF₂ crystalline microcavity also prove its potential in optical filter, cavity quantum electrodynamics, nonlinear optics, and optical gyroscope.