Results and Discussions As a proof of concept, a 4×4 optical matrix multiplication with 10 Gb/s and 20 Gb/s data rates is verified by the simulation platform Ansys Lumerical. Four wavelengths are used as the input pulse, which is a binary sequence composed of 0 or 1. The output value obtained by optical simulation has a high fit with the theoretical calculation value (Fig. 6). Then, the NVSP-CNN is compared with the DEAP-CNN structure in terms of the rate, area, power consumption, and accuracy. Similar to the case of DEAP-CNN, the computing rate of the NVSP structure is limited by the digital-to-analog converter (DAC) modulation rate. The highest operation rate can reach 5 GSa/s, which is faster than the operation rate of the mainstream GPU. Compared with DEAP-CNN, the proposed accelerator structure can reduce power consumption by 48.75% while maintaining the original operation speed, and the area at the matrix operation can be reduced by 49.75%. Finally, the simulations on the MNIST and notMNIST datasets are performed, and inference accuracies of 97.80% and 92.45% are achieved, respectively. The recognition results show that the accelerator structure can complete most image recognition tasks in life.

The convolutional neural network (CNN) has achieved great success in computer vision and image and speech processing due to its high recognition accuracy. This success cannot be separated from the support of the hardware accelerator. However, the rapid development of artificial intelligence has led to a dramatic increase in the amount of data, which places stricter requirements on the computing power of hardware accelerators. Limited by the power and speed of electronic devices, traditional electronic accelerators can hardly meet the requirements of hardware computing power and energy consumption for large-scale computing operations. As an alternative, micro-ring resonator (MRR) and Mach-Zehnder interferometer (MZI)-based silicon photonic accelerators provide an effective solution to the problem faced by electronic accelerators. However, prior photonic accelerators need to read the weights from the external memory when performing the multiply-accumulate operation and mapping each value to the bias voltage of the MRR or MZI units, which increases the area and energy consumption. To solve the above problems, this paper proposes a nonvolatile silicon photonic convolutional neural network (NVSP-CNN) accelerator. This structure uses the Add-Drop MRR and nonvolatile phase change material Ge₂Sb₂Te₅ (GST) to realize optical in-memory computing, which helps improve energy efficiency and computing density.

Firstly, we design a photonic dot-product engine on the basis of GST and the Add-Drop MRR (Fig. 2). The GST is embedded on the top of the MRR, and its different crystallization degrees are used to change the refractive index of the MRR, which makes the output power of the Through and Drop ports change. The crystallization degree of GST is modulated outside the chip, and the light pulse increases the internal temperature of GST to change the crystallization degree. It is then cooled rapidly so that the crystallization state is preserved. This value remains unchanged for a long time without external current. During computational operations, a short and low-power optical pulse is injected from the MRR's input port and output from the Drop and Through ports. The output optical power is converted to electric power through a balanced photodiode, and Td-Tp is realized in the meantime. Therefore, the values of Td-Tp under different GST phase states can be used as the weight values in the neural network (Fig. 3). Then, we propose an optical matrix multiplier combined with wavelength division multiplexing (WDM) technology and the GST-MRR-based photonic dot-product engine (Fig. 4). Finally, the optical matrix multiplier is combined with the nonlinear parts (activation, pooling, and full connectivity) to build a complete accelerator, i.e., the NVSP-CNN accelerator (Fig. 5). In NVSP-CNN, the convolution operation is implemented optically, and the nonlinear part is realized electrically.

This paper proposes an MRR and GST-based photonic CNN accelerator structure for in-memory computing. Unlike the traditional MRR-based accelerator, the NVSP-CNN accelerator can avoid the power loss caused by the continuous external power supply for state maintenance and does not require external electrical pads for modulation. Hence, it can effectively reduce area loss. In addition, we implement the simulations on the MNIST and notMNIST datasets and achieve inference accuracies of 97.80% and 92.45%, respectively. Therefore, the proposed structure has advantages in power consumption, area loss, and recognition accuracy, which is expected to tackle most image recognition tasks in the future.