Aimed at the problem of modulation recognition in wireless communication, a lightweight CNN modulation recognition algorithm based on convolutional neural network (CNN) is proposed. The spatial features of the signal were extracted by continuous multiple convolution operations, and features were dimensionally mapped by fully connected layer. The recognition probability was output by using the Softmax layer to achieve the recognition of multiple modulation signal. The trained lightweight CNN model was compressed by X-CUBE-AI and deployed on the STM32F405RGT6 embedded microcontroller, and the RADIOML2016.10a data set was used to test the overall performance of the deployed model. The experimental results show that the designed lightweight CNN model only occupies 1474.6 KiB Flash and 150 KiB RAM, which has fewer parameters than other deep learning models. When the signal-to-noise ratio is 0 dB and above, the recognition accuracy is up to 81.8%. The model can achieve modulation recognition effect comparable to PC platform on ARM Cortex-M embedded platform, which verifies the effectiveness and feasibility of the algorithm.