Abstract

a Visual perception process of the kingfisher, and a schematic illustration of its visual system. b Schematic diagram and working principle of the PSOS device. c Schematic of biological unipolar neurons and synapse interconnecting pre- and post-neurons. d Demonstration of feature extraction and motion recognition based on PSOS devices.

In addition, to explore the ability of polarization-sensitive synaptic electronics to extract effective information in a complex environment, a physical scene of fish trajectory recognition under a dazzling water surface is constructed. Reservoir computing (RC) systems is constructed using the polarization-state-dependent postsynaptic currents (STP) for time derivation training, and an artificial neural network (ANN) based on the LTP is used to distinguish the direction of fish movement under a dazzling water surface. This method only needs to train the read function, and can effectively calculate complex time related data with low training cost (Fig. 1d). Leveraging its synaptic and polarization-sensitive properties, the PSOS device are expected to have excellent feature extraction capabilities and enable precise recognition of motion trajectories in complex environments.

Material and synaptic characteristics of PEA₂SnI₄ microwires

The PEA₂SnI₄ microwires arrays is prepared by template-assisted imprinting method (Supplementary Fig. 2, and the detailed fabrication procedure of the devices is described in Methods), which offers distinct advantages, including precise control over microchannel dimensions, large-area fabrication capability, and excellent pattern scalability. Here, a flexible PSOS 10 × 10 array device using the template-assisted imprinting method is prepared, the corresponding photograph is shown in Fig. 2a. The optical image in Supplementary Fig. 3 indicates that the microwire array exhibits highly consistent linewidth and periodicity over a large area, forming a regular grating structure with clear directionality. To verify the quality of the microwire array, detailed characterizations are conducted. First, the morphology of large-area prepared PEA₂SnI₄ microwires is observed using scanning electron microscopy (SEM) measurement, and the result in Fig. 2b shows that the surface of microwires is smooth and dense, with excellent uniform crystallinity. And the cross-sectional image reveals that the thickness of microwires is approximately 1 μm, with consistent height and width throughout. Then, the corresponding elemental distribution is analyzed by energy dispersive spectrometer (EDS), which shows that the atomic ratio of Sn and I elements is 1: 4, in perfect agreement with the stoichiometric ratio (Supplementary Fig. 4). The gaps between the microwires indicate the absence of excess residues, thus verifying the successful and precise synthesis of the PEA₂SnI₄ microwires array, enabling the structural anisotropy. At the same time, atomic force microscopy (AFM) is employed to characterize the surface roughness of the microwires. The three-dimensional (3D) image in Fig. 2c clearly demonstrates the high quality of the microwires array, the surface roughness is 8.63 nm. The 2D projection at the bottom further confirms the homogeneous spacing between the microwires. Furthermore, the X-ray diffraction (XRD) result reveals the sharp and intense (0 0 n) diffraction peaks, indicating that the microwires possess a pure phase and high orientation (Fig. 2d). The selected area electron diffraction (SAED) pattern, shows distinct and orderly diffraction spots, which confirm the exceptional single-crystal quality of the synthesized microwires. Notably, the SAED results show the diffraction points corresponding to the (0 0 8) and (0 1 0) lattice planes, which is consistent with the XRD measurement results, indicating that the microwires have a highly preferred orientation along the (0 0 n) plane. Furthermore, the crystal structure of PEA₂SnI₄ is shown in Fig. 2e, where the PEA⁺ (organic sheet) vertically surrounds the inorganic layer to form a 2D perovskite layered structure, which consists of individual Sn²⁺/I⁻ octahedral with shared angles. Moreover, density functional theory (DFT) simulations are employed to calculate the surface energies of PEA₂SnI₄ along three distinct crystal planes of (1 0 0), (0 1 0), and (0 0 1) (Fig. 2f). According to the Supplementary Note 1, the calculating results indicate that the (0 0 1) plane exhibits the lowest surface energy, suggesting a preferential growth orientation along this plane and the strong stability during crystal formation. This result is consistent with the above XRD measurement. Due to the morphology anisotropic and the low-symmetry lattice structure, PEA₂SnI₄ microwires with one-dimensional (1D) structure is particularly advantageous as light-absorber elements for polarization-sensitive synaptic device.

Fig. 2: Material characterization and synaptic properties of the device.

a Photograph of a large-scale array device (scale bar: 1 cm). b Surface and cross-section (inset) SEM images of microwires. Scale bars, 20 µm. c AFM image of PEA₂SnI₄ microwires with smooth surface, sharp edge, and homogeneous size. d XRD pattern and SAED pattern (inset) of PEA₂SnI₄ microwires (scale bar: 2 nm^–1). e The crystal structure and (f) surface energy magnitudes of PEA₂SnI₄ along (1 0 0), (0 1 0) and (1 0 0) crystal planes. g PPF index expressed as a function of optical pulse widths. Inset: The ΔEPSC stimulated by two consecutive optical pulses with a fixed duration and interval of 0.5 s (520 nm, 7.56 mW cm^–2). h Final current values (defined as the 7.5th second) of the reservoir state with optical pulse stimulation of 16 combinations. The value represents the average obtained from testing ten devices under identical conditions. i Cycling endurance of LTP and LTD across 6 consecutive cycles.

To gain insights into the synaptic and photoelectric performance of the device, a series of investigations are conducted. As shown in Supplementary Fig. 5, when illuminated with light (520 nm, duration of 0.5 s), the current increases to 2.06 μA. After the light is turned off, the current shows a rapid decay followed by a slow decay before stabilizing at a constant level. After 1000 s, the current remains 0.21 μA higher than the initial value, indicating that the device has the synaptic-like dynamic regulation capabilities. To evaluate the photoelectric performance of the device and its potential for neuromorphic computing applications, we investigate its photosensitive characteristics. The light duration time is fixed at 0.5 s. As illustrated in Supplementary Fig. 6, the device exhibits a discernible EPSC response even under a weak illumination of 3.10 μW cm⁻². Moreover, the EPSC curves under different light intensities can be clearly distinguished. The responsivity (R) is also calculated based on the instantaneous current upon light termination. Remarkably, even under weak illumination of 3.10 μW cm⁻², the device achieves an R of 11.8 A W⁻¹, demonstrating its promising potential for practical application. To further verify the large-area uniformity and functional consistency of the device, photoelectric tests on the 10 × 10 array are performed. Supplementary Fig. 7a reveal that the dark current stabilizes around 8 µA, indicating a uniform background current. Upon applying a single optical pulse and five consecutive pulses (duration of 0.5 s), the photocurrent responses across the array remain highly uniform, all the pixels have similar dark current (8.01 ± 0.019 µA) and photocurrent (8.43 ± 0.006 µA, 1 pulse; 8.64 ± 0.013 µA, 5 pulses), which prove that the array possess satisfactory uniformity (Supplementary Fig. 7b).

PPF is an important form of STP, reflecting the enhanced postsynaptic current amplitude in response to two successive spikes²⁴. As shown in the inset of Fig. 2g, by consecutively applying two identical light pulse stimuli, the PSC value did not return to its initial state before the second stimulus. This is due to the fact that the photogenerated carriers trapped by the Sn vacancy cannot be completely released instantaneously²⁵. Upon the arrival of the second optical pulse, more photogenerated carriers induce a higher current level, so this accumulation triggers the PPF effect. To quantify the PPF index induced by EPSC, it is defined as the ratio of the amplitude of the second PSC (A₂) to the first PSC (A₁)^26,27. Under 520 nm illumination, a progressive decline in the PPF index is observed with increasing time intervals between the two stimuli (Fig. 2g). When the interval was 0.05 s, the maximum PPF index reached 120%.

Another key characteristic of synaptic devices is LTP. Generally, the transition from STP to LTP occurs through prolonged and repeated stimulation, which serves as the foundation of biological learning and memory. Herein, the ΔEPSC behavior varies in response to light pulses with different intensities, different pulse width and light pulse numbers, as shown in Supplementary Fig. 8a−c. It can be observed that with the stimulus enhancement, the ΔEPSC value gradually increases after the light is removed, producing a transition from STP to LTP. Remarkably, even after 1000 s, the current maintains a level higher than the initial baseline as light stimulation intensifies, confirming its current retention capability. This phenomenon is consistent with the dependence of neurotransmitter concentration changes within biological synapses²⁸. In detail, the human visual system gradually forgets optical stimuli after perceiving and memorizing them in real time, and multiple rehearsal processes are typically required to consolidate the memory²⁹. Therefore, we explore the learning-forgetting-relearning curve of the synaptic device correspondingly, as shown in Supplementary Fig. 9. After applying 20 light stimuli during the first learning phase, followed by a 30 s forgetting process, only 17 stimuli cycles are needed to reach the same level of learning again. This result indicates that the synaptic device could simulate the learning and memory process of the human brain.

Remarkably, the light response of PSOS remains nonlinear even when stimulated by multiple pulses, which is key in classification and recognition tasks in biological activity. Therefore, we construct a nonlinear mapping feature space using the nonlinear relationship between the electrical conductivity of the device and the external light stimuli. First of all, the separability of reservoir states generated by the synaptic device is verified by sequentially applying 4-bit pulse sequences of different light pulse inputs to the device. In this context, each optical pulse input (0.25 s pulse interval, 0.25 s pulse width) is considered as one bit, respectively, where “0” and “1” represent the off and on states of the optical pulse. The 4-bit encoding encompasses a total of 16 distinguishable pulse sequences from “0000” to “1111”. Here, we randomly selected 10 devices for 4-bit reservoir testing. As shown in Supplementary Fig. 10a−j, all tested devices exhibit reproducible current variation curves, which originate from the high uniformity of the single-crystal microwire arrays fabricated by the template-assisted imprinting method. The PSC at the 7.5th second is chosen as the final state. Notably, it can be found that the final state of the reservoir is closely related to the number and sequence of the light pulse stimulation, and the all 16 encoding sequence states can be effectively distinguished. The standard deviation of the final states across all devices and sequences are summarized in Supplementary Fig. 11. The results demonstrate that the standard deviations have no overlapping parts and there is relatively obvious difference under different stimulation sequences. For rigorous quantitative analysis, we adopted the averaged current values from ten devices under different sequential stimuli as the final state currents, thereby validating the separability of reservoir states generated by the synaptic devices (Fig. 2h). Therefore, the device has the feature mapping capability, which is conducive to the subsequent feature extraction.

Finally, we simulate IPSC behavior in biology by applying reverse voltage (polarity opposite to the read voltage), which is also typical of changes in synaptic weights in learning and memory of organisms. Supplementary Fig. 12 displays the influence of regulatory synapse on the relaxation process, in which a series of 100 consecutive reverse voltage pulses is applied (− 1.0 V, 5 Hz) after removing light (4.82 mW cm^–2, 5 Hz) at a steady operating voltage of 0.1 V. The results are compared with a control group where no reverse voltage pulses are applied. It is observed that the relaxation current returns to its initial state significantly faster upon applying the reverse voltage pulses, providing clear evidence of electrical erasure behavior. In contrast, the curve without negative voltage stimulation demonstrates the long-term retention characteristics of the device, where it does not recover to the initial dark current state within a short period. To further investigate the potential application in simulating neuromorphic computing within human visual systems, the LTP and long-term depression (LTD) behaviors are explored. By consecutively applying 100 light pulses (520 nm, 21.62 mW cm^–2, 0.05 s pulse width, 0.15 s interval) followed by 100 reverse electrical pulses (− 1 V, 0.1 s pulse width, 0.1 s interval), the conductance of the device changes regularly with the applied pulse number (Fig. 2i). The cycle-to-cycle variation of the LTP and LTD characteristics of the device is also investigated under the same condition. The current variation range of the device remains nearly identical across six cycles, and the current potentiation and depression processes in each cycle exhibit good repeatability, with no observable degradation. To verify the retention capability of the current states induced by both light and electric stimuli, we conduct retention measurements on 5 states following both optical and electrical programming operations (Supplementary Fig. 13). It can be observed that the current exhibits an initial rapid decay followed by a slower decay before stabilizing at a constant level. The phenomenon is consistent with other optoelectronic synapses^30,31,32. All states remain clearly distinguishable even after 1000 s, and the current maintains a level higher than the initial baseline as light stimulation intensifies, confirming its current retention capability. These findings prove that the device not only enables short-term modulation but also holds promise for brain-like long-term memory functionality. In addition, its conductance tuning characteristics demonstrate good linearity with stable weight updates. This characteristic is crucial for ANN training, indicating the broad application prospects and significant development potential in the field of neuromorphic computing.

Polarization dependent response of microwire synapse array

To study the anisotropy of PEA₂SnI₄ microwires arrays, we construct a custom optoelectronic testing system, as illustrated in Fig. 3a. By rotating a half-wave plate, the polarization direction of the incident light is adjusted easily, and the typical response curves with incident light of different polarization angles can be obtained. To investigate the polarization-dependent optical properties, we first perform the polarization-dependent absorption spectrum measurements. It can be observed that within the wavelength of 600 − 700 nm, the absorption spectra from 0° to 180° exhibit highly overlap, and the peak intensity fluctuates accordingly with changes in polarization angle (Fig. 3b). The diagram in Fig. 3c clearly reveals the spindle-shaped feature of the polarization distribution. The curve can be fitted by a sine function with a period of 2π, and the absorption intensity shows a polarization ratio of ~1.1: 1, indicating that the microwire arrays has anisotropy characteristics.

Fig. 3: Analysis of polarization-dependent detection capabilities.

a Schematic diagram of the testing setup. b Normalized absorption spectra of PEA₂SnI₄ microwires at different angles. c Polar diagram of anisotropic absorption values of microwires at different polarization angles at 647 nm. d ΔEPSC of the device stimulated by 30 pulses at 0° and 90° polarization angles (520 nm, 7.56 mW cm⁻²). e Dynamic response of ΔEPSC to five consecutive pulses under different polarized light stimulations (520 nm, 2.26 mW cm^–2) and (f) the corresponding peak polar diagram. The simulated polarization imaging landscape map at (g) 0° and (h) 90° polarization angles processed by the imaging system, and (i, j) corresponding histograms of the corresponding pixel distributions for each output image.

To gain a deeper understanding of the impact of polarized light on the synaptic performance of the device, we utilize the system to evaluate the response under different polarization angles. Here, the polarization angle parallel to the microwires is defined as 0°. From Fig. 3d, it is obvious that the value of EPSC at a polarization angle of 0° is larger than that at a polarization angle of 90°, which is consistent with the absorption results. To explore the relationship between polarization-perceptual EPSC and polarized angle, five consecutive light-pulse stimuli are applied under different polarization angles (0° − 360°, intervals of 30°) (Fig. 3e). It can be clearly seen that the EPSC has a significant polarization dependence. By statistically analyzing the peak current of the fifth pulse, a polar coordinate diagram is plotted to clarify the polarization-dependent degree (520 nm, 2.26 mW cm^–2), as shown in Fig. 3f. The result shows that the dichroic ratio reached up to 1.38, fully demonstrating the polarization-sensitive detection capability of the device. A comparison of the parameters between the proposed device and other polarization-sensitive synaptic devices is listed in Supplementary Table 1. The prepared device shows advantages in ΔEPSC* (defined as |I_0° − I_90°| at fixed light intensity), read voltage and array size. Based on this, a 3 × 3 device array is used to conduct imaging experiments of letters under different polarization states, as shown in Supplementary Fig. 14a, b. A “T”-shaped shadow mask is used to apply multiple light pulses with varying polarization angle to the synaptic device array, and the corresponding current changes are recorded (Supplementary Fig. 14c, d). For imaging under the same polarization states, the results reveal that the clarity of the letter images improved significantly with the increase of pulse number. For imaging under different polarization states, with the increase of the number of pulses, the difference between the letter image at 0° and 90° polarization becomes more obvious. This phenomenon is primarily attributed to the unsaturated EPSC behavior of the devices under multi-pulse stimulation, which amplifies the differences in polarization effects. Notably, the brightness contrast of the letters is significantly enhanced under the 0° polarization state, further demonstrating the critical role of polarization synapses in improving the contrast of polarized imaging.

Leveraging the imaging properties of the device under different polarization states, it holds significant potential for reducing reflected light interference, thereby enhancing imaging quality. This feature is valuable for application scenarios such as autonomous navigation and surveillance systems, where accurate and efficient object recognition is essential. Here, the polarization-state-dependent synaptic currents at different polarization states (0° and 90°) are employed to extract fish information under the dazzling water surface, the results are illustrated in Fig. 3g, h. It indicates that with the increase of the light pulses number, the image contrast is gradually improved, and the contour of the fish is progressively clearer. Notably, the contrast of images simulated under the 0° polarization state is significantly higher than that under the 90° polarization state, further demonstrating the pivotal role of the polarization synaptic device in extracting effective information in a complex environment. To better visualize the results, the pixel distribution histograms of each grayscale image are summarized, as illustrated in Fig. 3i, j. It is noteworthy that as the number of pulses increases, the pixel distribution range expands, indicating that more effective information can be extracted from the images. Compared to 90°, the image pixel distribution at 0° is wider, further demonstrating that polarization synaptic can reduce reflections and enhance image feature information.

Feature extraction of static images

Considering the polarization-state-dependent synaptic properties of the device, the potential applications of the PSOS device in image convolution processing can be expected. Convolution involves element-wise multiplication and summation of a convolution kernel with image pixels. By assigning specific weights to the convolution kernel values, it is adaptable for diverse image processing tasks. Herein, to validate the ability of the device to extract features in different modes, by taking advantage of the STP properties of devices under different polarization states, two 3 × 3 convolutional kernels of [−?900?90−?00?0−?900?90] and [−?90−?0−?90000?90?0?90] are constructed and used for image recognition and edge extraction tasks. In detail, the responsivity of the device at polarization angles of 0° and 90° is multiplied and summated with the original image pixels, and the resulting image after edge extraction is obtained. Notably, the constructed convolutional kernels are able to extract key features in the vertical and horizontal directions, respectively (as shown in Fig. 4a). Figure 4b shows the identified results of the images after multiple convolution operations on the original building image by using different kernels. It can be seen that the vertical and horizontal features of the original image are efficiently recognized. By overlaying the two images after feature extraction, we obtain the edge feature map, which clearly shows the outline of the original grayscale image. The obtained result is basically consistent with the recognition result under software-defined edge detection conditions (Supplementary Fig. 15), verifying the effectiveness of the device in feature extraction. Furthermore, 50% salt pepper noise is implemented to the original image, and the feature extraction results are presented in Supplementary Fig. 16. Under the condition of 50% salt pepper noise, the software-defined edge detection fails to recognize the building image, while the image can be effectively recognized using the constructed convolutional kernels based on the device.

Fig. 4: Image feature extraction and recognition using polarization-state-dependent convolutional kernels.

a Schematic diagram of artificial visual preprocessing using polarization-state-dependent convolutional kernels and a three-layer ANN based on the synapse. b Feature extraction of the input original building image in vertical, horizontal, and edge modes according to the STP-defined edge detection (scale bar: 10 m). c Recognition images of handwritten digit “6” in the MNIST dataset using the STP-defined edge detection (vertical feature extraction) before applying noise and (d) after applying 50% salt pepper noise, respectively. e Equivalent circuit diagram of a 3 × 3 convolution kernel based on a 3 × 4 microwire array and (f) the corresponding outputs after hardware convolution operations. g The recognition accuracy under different information processing conditions. h The confusion matrixes by ANN with convolutional kernels and added noise.

To explore broader applicability, four convolutional kernels (vertical, horizontal, and two diagonal orientations) are used to extract features from handwritten digits in the MNIST database. Notably, diagonal kernels contribute to enhancing the sensitivity of the system to slanted edges and complex structures, which is beneficial in the recognition of handwritten characters and other irregular patterns. Taking the digit “6” as an example, the sharpened results are shown in Fig. 4c and Supplementary Fig. 17. These results confirm that the feature extraction performance based on the device is comparable to software-defined edge detection, consistent with earlier observations. After adding 50% salt pepper noise, feature extraction results (Fig. 4d and Supplementary Fig. 18) demonstrate that the device-based kernels retain their feature recognition capability, whereas the software-defined methods failed. This compelling evidence highlights the superior noise tolerance of the polarization-sensitive device with time-evolved-based convolutional kernels.

The aforementioned polarization-dependent convolutional kernel method employs software assistance to achieve feature extraction from complex images. This functional implementation relies on illuminating the device with polarized light and modulating polarization angles using external polarizers or waveplates to acquire multiple polarization responses, requiring complex optical and electronic systems. In order to effectively reduce the complexity of the circuit and verify the possibility of the practical application of polarization state-dependent convolution kernel hardware, we successfully created horizontal and vertical orientations microwire arrays with polarization selectivity. This process benefits from the advantage of the template-assisted imprinting strategy, that different patterns can be predetermined. The schematic diagram of the convolution kernel defined by a 3 × 4 microwire array is shown in the inset of Fig. 4e. The horizontal and vertical microwire arrays can sense polarized light at 0° and 90°, respectively. The corresponding SEM images (Supplementary Fig. 19) confirm that the microwire arrays exhibit a dense and smooth morphology across different orientations. Subsequently, image convolution operations are performed by moving the convolution kernel across the target image. Here, a 10 × 10 pixel letter “T” is selected as the feature extraction target, with the specific circuit configuration shown in Fig. 4e. The output results (Fig. 4f) demonstrate that the designed convolutional hardware achieves feature extraction of target objects without requiring complex polarizers or logic circuits. This intrinsic design eliminates the need for polarization-tuning optics, simplifying system integration and reducing implementation complexity.

Subsequently, based on the feature extraction capability of the convolution kernels, an ANN is further constructed based on the LTP of polarization-sensitive synaptic devices for image recognition. Here, ten handwritten digit images from “0” to “9” in the MNIST database are first preprocessed through the vertical and horizontal convolutional kernels, and these preprocessed results are then passed to the input layer, hidden layer, and output layer of the neural network for image training and recognition. The hidden layer consists of neurons with activation functions to enhance the nonlinear mapping capability of the network. Further simulation details of the ANN are provided in the Artificial Neural Network Simulation section. The detailed definitions of synaptic weights and related parameters are provided in Supplementary Note 2. In order to deeply explore the effect of the convolutional kernel on the improvement of ANN recognition ability, we employ the backpropagation algorithm to iteratively update the weights for the MNIST dataset with pixel dimensions of 28 × 28. As shown in Fig. 4g, the ANN incorporating convolutional kernels exhibits higher recognition accuracy and faster convergence speed, and the recognition accuracy of the device is increased from 90% to 97.6%, demonstrating the significant role of convolutional kernels in effectively extracting edge features. When 50% noise is implemented to the images, there is no significant decrease in the convergence speed or recognition accuracy, further indicating that the neural network based on this device has good fault tolerance and noise reduction ability after convolutional kernel preprocessing. In addition, during the training process of neural networks, the confusion matrix plays a crucial role for measuring classifier performance³³. Therefore, the confusion matrix is used to explore learning classification tasks. Similarly, ten handwritten digit images from “0” to “9” in the MNIST database are trained through three different neural networks, including ANN, the ANN incorporating convolutional kernels, and the ANN incorporating convolutional kernels with noise, and the confusion matrix classification results are shown in Fig. 4h and Supplementary Fig. 20, respectively. It can be found that the ANN with convolutional kernel preprocessing has higher recognition accuracy and is less prone to errors, suggesting that convolutional kernel preprocessing can effectively extract relevant feature information, reduce redundant data and noise, and facilitate subsequent image recognition processes.

To further validate the feasibility of hardware kernel implementation, we constructed a 4 × 4 microwire synaptic device array capable of performing simple multiply and accumulate operations convolution operations directly in hardware. Each device in the array supports weight modulation under optoelectronic cooperative control. As shown in Supplementary Fig. 21a and Fig. 21b, the intensity values of 2 × 2 pixels from the input image (Supplementary Fig. 21c) are first converted into voltage signals (0 or 0.1 V), which are then applied to each row of the microwire synaptic array. Within this array, two columns serve as a differential pair to represent the positive and negative weights, and the microwire array is programmed to the desired weights through parallel programming. Through this process, the output current between the positive and negative weight columns is computed as the sum of the products of the channel conductance and the input voltage. In the absence of input images, a voltage of 0.01 V is applied, and the differential output current is processed using a subtractor circuit composed of two operational amplifiers. The inverting inputs of the first and second operational amplifiers are respectively linked to the columns representing positive and negative weights. For the 2 × 2 pixel input image, the ANN multiply and accumulate operation based on the microwire synaptic array and operational amplifiers produces different current outputs according to the image features, as shown in Supplementary Fig. 21d. These results demonstrate that the hardware constructed from the synaptic array enables parallel processing of simple operations.

Motion recognition of dynamic images

To explore the practical application of polarization-sensitive perception technology, a physical scenario is constructed to recognize the trajectories of a small fish under complex light conditions. As previously discussed, Supplementary Fig. 10 shows that all 16 encoding sequence states based on the synaptic device can be effectively distinguished, indicating its potential as an in-sensor reservoir. This reservoir can map complex image inputs to reservoir states for feature extraction, and the ANN including one input layer, one hidden layer and one output layer can classify according to these features³⁴, as illustrated in Fig. 5a. The reservoir captures the movement trajectories of underwater fish using a custom dataset and is trained to recognize four directions of movement: up, down, left, and right within the plane. To predict the movement of the fish, a behavioral model of the device is constructed to characterize its response current to optical pulse stimuli. As shown in Fig. 5b, the current response of the device to a single optical pulse under different polarization states is displayed, with I_d gradually relaxes after the light is removed. The relaxation time is obtained by fitting the behavioral curve with a stretched exponential expression³⁵:

where the current increment induced by the light stimulus is defined as ΔI_d, and β (β < 1) is the inhibitory temporal factor. Both τ and β are obtained by fitting Eq. (1), as shown in Fig. 5b. Subsequently, the variation of ΔI_d with I_d for each individual light pulse is investigated under different polarization angles, as shown in Fig. 5c. The magnitude of ΔI_d largely depends on the current level of I_d. The behavioral curve is then fitted based on the following formula:

By substituting the decayed current value from Eq. (1) as I_d in the subsequent pulse into Eq. (2), we can obtain ΔI_d under the next pulse stimulation. Bringing this ΔI_d back into Eq. (1), we can derive the current value at any time after the stimulation. In this manner, the temporal pixel evolution is embedded into the conductance changes of a single pulse in the device, and these consecutive frames are linked together to represent the temporal features of the video.

Fig. 5: Motion recognition based on temporal dynamics characteristics of the PSOS device.

a Schematic diagram of the in-sensor RC system for categorizing the movement trajectories of underwater fish. b Current response to a single optical pulse (520 nm, f = 5 Hz, 0.05 s) and (c) experimental result of the ΔI_d − I_d relation under different polarization states. d Schematic diagram of motion trajectories mapping spatiotemporal vision information. e Schematic trajectory of underwater fish movement in four different states. The confusion matrix for classification of motion trajectories at (f) 0° and (g) 90° polarization states. h The action recognition accuracy under different polarized states. i The probability of the four possible trajectory outcomes after inputting image samples of motion under different polarization states into the processed training network. The dimensionality reduction of the reservoir outputs using LDA at (j) 0° and (k) 90°polarization states.

To fully exploit the spatiotemporal information of consecutive frames for trajectory prediction, we divide the fish’s movement into four consecutive frames, representing four different time points (t₀, t₁, t₂, t₃) during the motion process. Figure 5d illustrates the motion trajectory of the fish, from which four frames are extracted. To clarify the dynamic process, Fig. 5e provides a schematic representation of the fish trajectory at four different motion states, offering an intuitive understanding of the spatiotemporal evolution. Each frame is converted into a single-channel grayscale image with gray values ranging from 0 to 255, followed by binarization through setting an appropriate threshold. By connecting adjacent frames, a pulse sequence is constructed for each pixel, mapping the spatiotemporal visual information, which is then input into the in-sensor reservoir. Subsequently, a fully connected network with hidden layers and the backpropagation algorithm are employed for readout processing. Then, we utilize continuous frame images containing temporal information to classify motion outcomes. Figure 5f, g clearly present the confusion matrixes obtained from classifying motion trajectories under 0° and 90° polarization states. The matrixes are dominated by diagonal elements, which fully demonstrates the model’s excellent performance in predicting accuracy when assessing different input directions. In particular, under the 0° polarization state, the model exhibits high probabilities along the diagonal elements and low probabilities of incorrect predictions, indicating higher accuracy in determining motion directions. Figure 5h shows the changes in training accuracy during the iteration process under 0° and 90° polarization states. The experimental results indicate that recognition is faster and more accurate under the 0° polarization state. After superimposing these four frames, the trajectory contour of the fish throughout its movement becomes clearly visible, successfully capturing its motion trajectories in different directions. Using this bionic ANN for training and learning not only allows for the acquisition of spatial information but also outputs temporal information for motion direction recognition by compressing the number of frames³⁶. Notably, the superposition effect of the fish’s motion trajectory under 90° polarization is less significant than that under 0°, highlighting the effectiveness of polarization-sensitive photoelectric synapses in reducing the impact of glare in bright light environments. To further validate the higher recognition efficiency of the 0° polarization state, we randomly select 20 image samples with different motion directions and input them into the ANN. The probabilities of the four possible outcomes for each motion trajectory are then calculated (Fig. 5i). Compared to the 90° polarization state, the probability of correctly perceiving the motion trajectory under the 0° polarization state is significantly higher, further confirming the superiority of the 0° polarization state in motion trajectory recognition. Subsequently, we use linear discriminant analysis (LDA) for dimensionality reduction and further plot 2D clustering maps for the recognition of fish motion directions under different polarization states after training. Among them, different motion trajectories are represented by different colors, and the “x” and “y” axes correspond to the new 2D feature vectors after dimensionality reduction. Figure 5j, k show the 2D clustering maps drawn using the feature vectors extracted by the photoelectric synapses under 0° and 90° polarization states, respectively. By comparison, it is found that after training, the distribution of each motion direction is clearer and more distinct under 0° polarization compared to 90° polarization, indicating the model’s exceptional performance in reducing the impact of glare and accurately distinguishing motion directions in bright light environments. This polarized synaptic vision system is expected to provide more complex application scenarios in imaging, military reconnaissance, and remote sensing detection.