A fast and robust fundamental matrix estimation method based on Sequential Similarity Detection Algorithm (SSDA) is presented to estimate the fundamental matrix rapidly and accurately. The SSDA is introduced into the Maximum a Posteriori Sample Consensus (MAPSAC) to search the optimum model parameters and the accumulation times of computing a cost function are cut down by eliminating the false model as soon as possible, which not only keeps the better robustness of MAPSAC, but also reduces its computation effectively. Then, the initial inliers obtained by the improved MAPSAC are optimized with a M-estimator. Those inliers with larger residual errors are removed and the optimized inliers are used to compute the fundamental matrix to enhance the precision and improve the robustness of the algorithm. Experiment results demonstrate that the proposed algorithm performs better in accuracy and robustness, and its average speed has increased at least 30% as compared with that of the MAPSAC. The proposed algorithm can satisfy the requirements for real-time, precision and robustness in the fields such as three-dimensional reconstruction, image matching, image tracking and camera self-calibration.