The collaborative application of point cloud data and optical remote sensing image has been widely concerned in the field of remote sensing. In order to accurately register two kind of data and better integrate their advantages， an automatic registration method of point cloud and optical remote sensing image in urban scene is proposed. Firstly， the depth image is generated from point cloud data， that is， 3D data is converted into 2D image. Secondly， the Unet model is used to train the depth image and the optical remote sensing image respectively and get building segmentations. Thirdly， the minimum circumscribed rectangles of buildings are constructed based on the contour set of building segmentation， and the length-width ratio of rectangle is taken as the constraint condition to find Corresponding Points（CPs）. Then， we use the similar triangle principle to find CPs of the rectangle’s center point. Finally， the coordinate of the CPs are substituted into the transformation model to calculate the model parameters， thus the registration is achieved. The experimental results show that the proposed method can achieve better registration effect when it is difficult to match with traditional point feature method， and it is resistant to image translation， rotation and scaling.