The existing infrared small target detection technologies have some shortcomings in terms of target detection capability, background suppression capability, and real-time performance, which fails to meet practical needs. Tensor analysis techniques have been widely used for infrared small target detection and have increasingly demonstrated superiority. However, three are three key issues, including suitable tensor structures, comprehensive tensor decomposition frameworks, and satisfactory real-time performance. Consequently, this paper proposes an infrared small target detection method based on four dimensional temporal-spatial tensor decomposition and block term decomposition-based norm (BTDN-4DST). Specifically, a four-dimensional sphered temporal-spatial image-patch tensor is firstly constructed to establish the data basis for tensor decomposition. Subsequently, a norm based on block term decomposition is defined to fully exploit the spatial-temporal characteristics of the background for accurate background estimation. Finally, an effective solution framework based on Alternating Direction Method of Multipliers is designed for solving the detection model. To validate the performance of the BTDN-4DST, six state-of-the-art infrared small target detection methods are selected as comparative algorithms, and extensive experiments and analyses are conducted on five real infrared image sequence datasets. BTDN-4DST can rapidly enhance the saliency of weak small targets and greatly suppress background and noise components, which proves that the proposed method not only excels in background suppressibility and target detectability but also exhibits satisfactory real-time detection performance, meeting practical application requirements.