The atmospheric boundary layer is the lowest layer of the troposphere, which is directly influenced by the surface. The atmospheric boundary layer height (ABLH) is an important parameter of the atmospheric boundary layer, whose value ranges from several hundred meters to thousands of meters. It plays an important role in analyzing the heat radiation transmission process in the boundary layer, acquiring the air pollution status, and formulating pollution control strategies. Lidar is an active remote sensing tool, which has high spatial and temporal resolutions and can continuously and automatically measure ABLH. The methods of estimating ABLH based on lidar data mainly include the threshold method, the gradient method, the wavelet covariance transform method, and the variance method. However, these methods are only suitable for specific meteorological conditions, and the interference of clouds or a suspended aerosol layer can easily lead to the misjudgment of ABLH. A highly robust ABLH estimation method combining K-means and entropy weight method, i.e., EK-means, is proposed to solve the problem of erroneous detection by commonly used lidar-based ABLH estimation methods under complex atmospheric structures. The proposed method improves the performance of ABLH estimation based on cluster analysis in terms of initial parameter selection and distance calculation. Compared with commonly used lidar-based ABLH estimation methods, the proposed method has a strong anti-interference ability. It can well track the diurnal variation process of the boundary layer under complex atmospheric structures. Under clear sky and cloudy weather or a suspended aerosol layer structure, the ABLH estimated by the proposed method is basically consistent with that measured by a radiosonde, and the correlation coefficient is 0.9718 and 0.9175, respectively. The proposed method has high robustness and can reliably estimate ABLH under different conditions.

The proposed method integrates K-means and entropy weight method to improve the ABLH estimation performance based on cluster analysis from two aspects of initial parameter selection and distance calculation. Firstly, a sample dataset is constructed depending on the characteristics of the boundary layer, the free troposphere, a cloud layer, and a suspended aerosol layer. Then the utility function is introduced, and the entropy weight method is used to calculate the weight attributes of sample features. Next, the initial parameters of K-means are determined. The number n of intervals in the same direction is obtained by analyzing the gradient of the lidar backscattering signal, and the number of clustering categories (k=n+1 or k=n+2) can be obtained for different conditions. The initial center of clustering is selected as the position of the maximum signal intensity in the intervals in the same direction. Two centers are evenly selected in the first negative interval, and the Davis-Bouldin index is used for fine tuning. Finally, the ABLH is estimated with category features, which is located at the category boundary seeing the first decrease in the clustering strength from bottom to top.

To assess the validity of the proposed EK-means, this paper uses the lidar data over Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) central facility (C1) to estimate ABLH under various conditions. Experiments show the comparison results of the diurnal variation of ABLH tracked by four methods under the conditions of clear sky, polluted weather, and cloudy weather or a suspended aerosol layer structure (Figs. 5-7). The improved K-means and the proposed EK-means can reliably track the diurnal variation process of ABLH under these three conditions, and the proposed EK-means has the best performance (Figs. 5-7). The gradient method and the wavelet covariance transform method are susceptible to complex atmospheric structures such as clouds or a suspended aerosol layer, and the tops of clouds or the suspended aerosol layer is estimated as the ABLH, which has a large error (Fig. 7). Experimentally, the paper also compares the ABLHs estimated by the four lidar-based methods and by the radiosonde under clear sky and cloudy weather or a suspended aerosol layer structure (Figs. 8-9). The ABLH estimated by the proposed method under clear sky and cloudy weather or a suspended aerosol layer structure is consistent with that measured by a radiosonde, and the correlation coefficients are 0.9718 and 0.9175, respectively [Fig. 8(d) and Fig. 9(d)]. The improved K-means also yields good experimental results with correlation coefficients of 0.9522 and 0.7986, respectively [Fig. 8(c) and Fig. 9(c)]. The ABLHs estimated by the gradient method and the wavelet covariance transform method are significantly different from that measured by a radiosonde under cloudy weather or a suspended aerosol layer structure, and the correlation coefficients are both less than 0.5 [Fig. 9(a) and Fig. 9(b)]. The proposed method has high robustness and can reliably estimate ABLH under different conditions (Table 1).

The experimental results show that the proposed method is a highly robust ABLH estimation method compared with other commonly used lidar-based ones such as the gradient method and the wavelet covariance transform method. The proposed method can better track the diurnal variation of ABLH under clear sky, polluted weather, and cloudy weather or a suspended aerosol layer structure. Under the conditions of clear sky and cloudy weather or a suspended aerosol layer structure, the ABLH estimated by the proposed method has better consistency with that measured by a radiosonde, having a higher correlation coefficient and a smaller mean absolute error.