Deep learning based high dynamic range (HDR) image processing algorithms has the problem of skin color deviation when processing images containing human figures. In response to this issue, this article proposes a portrait HDR image processing algorithm based on multi feature fusion-U²HDRnet. This algorithm consists of three parts: skin feature extraction module, trilateral feature extraction module and color reconstruction module. Firstly, the skin feature extraction module separates the color and position information of the skin region. Secondly, the trilateral feature extraction module extracts local features, global features and semantic features of the image, and fuses them with skin features. Finally, the color reconstruction module interpolates the grid in terms of space and color depth. In addition, this article adds an improved fusion module of self attention and convolution to improve the processing performance of HDR. At the same time, this article also produces the PortraitHDR dataset for portraits, filling the gap in the dataset in this field. The test results show that the PSNR of U²HDRnet reaches 31.42 dB, and the SSIM reaches 0.985, both of which are superior to the commonly used HDR algorithms. They obtain high-quality portrait HDR images while avoiding skin distortion.