Free-space optical communication has been an active research area and has gained significant interest due to its unique advantages, such as large bandwidth and high transmission capacity.1
Advanced Photonics, Volume. 5, Issue 4, 046009(2023)
Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding Author Presentation
Free-space optical information transfer through diffusive media is critical in many applications, such as biomedical devices and optical communication, but remains challenging due to random, unknown perturbations in the optical path. We demonstrate an optical diffractive decoder with electronic encoding to accurately transfer the optical information of interest, corresponding to, e.g., any arbitrary input object or message, through unknown random phase diffusers along the optical path. This hybrid electronic-optical model, trained using supervised learning, comprises a convolutional neural network-based electronic encoder and successive passive diffractive layers that are jointly optimized. After their joint training using deep learning, our hybrid model can transfer optical information through unknown phase diffusers, demonstrating generalization to new random diffusers never seen before. The resulting electronic-encoder and optical-decoder model was experimentally validated using a 3D-printed diffractive network that axially spans <70λ, where λ = 0.75 mm is the illumination wavelength in the terahertz spectrum, carrying the desired optical information through random unknown diffusers. The presented framework can be physically scaled to operate at different parts of the electromagnetic spectrum, without retraining its components, and would offer low-power and compact solutions for optical information transfer in free space through unknown random diffusive media.
1 Introduction
Free-space optical communication has been an active research area and has gained significant interest due to its unique advantages, such as large bandwidth and high transmission capacity.1
In this work, we demonstrate a jointly optimized electronic-encoder neural network and an all-optical diffractive decoder model for optical information transfer through random unknown diffusers. This electronic-optical design (Fig. 1) is composed of a convolutional neural network (CNN) that encodes the input image information of interest (to be transmitted) into a 2D phase pattern, like an encrypted code, which is all-optically decoded by a jointly trained/optimized diffractive processor that reconstructs the image of the input information at its output plane, despite the presence of random unknown phase diffusers that are constantly changing/evolving (see Video 1). In addition to phase encoding of input information of interest, we also report that amplitude encoding can be used as an alternative scheme for communication through random unknown diffusers using a jointly trained diffractive decoder. Our all-optical diffractive information decoder consists of passive diffractive layers with a compact axial span of
Figure 1.Pipeline of the hybrid electronic encoder and optical diffractive decoder for optical information transfer through random unknown diffusers. (a) Schematic drawing of the presented diffractive decoder, which all-optically decodes the encoded information distorted by random phase diffusers without the need for a digital computer. (b) Workflow of the hybrid electronic-optical model: the electronic neural network encodes the input objects into 2D phase patterns and the all-optical diffractive neural network decodes the information transmitted through random, unknown phase diffusers.
2 Results
2.1 Design of a Diffractive Decoder with Electronic Encoding for Optical Information Transfer through Unknown Random Diffusers
Figure 1(b) shows the operational pipeline of the presented electronic-encoder and optical-decoder model for optical information transfer through random phase diffusers. A CNN-based encoder (see Appendix) was trained to convert any given input image to be transferred into a phase-encoded pattern, illuminated by a uniform plane wave. The corresponding optical field distorted by random unknown phase diffusers was then decoded by the diffractive network to all-optically recover the original image at its output field–of-view (FOV).
Sign up for Advanced Photonics TOC Get the latest issue of Advanced Photonics delivered right to you!Sign up now
First, we analyze the impact of the encoder CNN on the optical information transfer through unknown random diffusers present in the optical path, and quantitatively explore its necessity, as opposed to a diffractive decoder that is trained alone. In this analysis, we compared it against the architecture of our previous work,29,30 which was used to see amplitude objects through random diffusers using a diffractive neural network, as shown in Fig. 2(a). Without any encoder neural network present, this diffractive processor could see through random unknown diffusers after its training with hundreds to thousands of examples of random diffusers, successfully generalizing to see through new random diffusers never seen before. However, with the increase in the distance between the input objects and the random phase diffuser plane, a performance drop was observed, as shown in Fig. 2(c). Here we used the Pearson correlation coefficient (PCC) to assess the quality of the output images synthesized by the diffractive network as a function of the Fresnel number (
Figure 2.Comparison between all-optical diffractive networks and the hybrid electronic-optical models for transferring optical information through random unknown diffusers. (a) Schematic of a four-layer diffractive network trained to all-optically reconstruct the amplitude images of input objects through random phase diffusers without electronic encoding. (b) Schematic of a four-layer diffractive decoder with an electronic encoder jointly trained to decode the encoded optical image through random unknown diffusers. (c) The information transmission fidelity (PCC) of the two approaches (all-optical versus hybrid) as a function of the Fresnel number (
To break through this limitation and extend the capabilities of diffractive neural networks to communicate through random unknown diffusers, we jointly trained an electronic encoding neural network to collaborate with the all-optical diffractive decoder network for transferring optical information through random unknown phase diffusers, covering a much larger span of
Figure 3.Optical information transfer through random phase diffusers using electronic encoding and diffractive decoding. (a) Random phase diffusers with a correlation length
To demonstrate the efficacy of this trained hybrid electronic encoder and optical decoder for transferring optical information through random phase diffusers, we first compared its performance to that of a conventional lens-based image transmission system (see Appendix). The imaging results of the same objects through the same diffusers, as shown in Fig. 3(d), exhibited blurry intensity profiles of handwritten digits “2,” “7,” and “8” when captured by an ideal lens. These blurry images clearly illustrate the negative impact of the random phase diffusers within the optical path, making it impossible to recognize the transmitted images with the naked eye. In contrast, the jointly trained electronic-optical system, comprising an electronic encoder/front end and an optical diffractive decoder/back end, can successfully transmit and receive these images through unknown new phase diffusers, as shown in Fig. 3(c), indicating the strong resilience of the hybrid system against the distortions caused by random diffusers during the signal transmission process.
To shed more light on the generalization ability of the hybrid model, we further tested it with additional handwritten digits sampled from the test set that were never used during the training stage; these test objects were individually distorted by two newly generated diffusers that were never encountered during the training (termed as new diffusers), as shown in Fig. 4, left two columns. Additional results of this hybrid electronic-optical model for transferring new optical images of interest through random unknown phase diffusers that are constantly changing are shown in Video 1. All of these resulting output images are easily recognizable compared to the blurry counterparts of the ideal lens, revealing the generalization of the trained electronic-encoder and optical-decoder model to new objects and new random phase diffusers that have never been seen before.
Figure 4.Simulation results of the hybrid electronic-optical model for optical information transmission through unknown random phase diffusers.
To provide examples of “external generalization” to test objects of different types, we used binary gratings with
2.2 Impact of the Number of Diffractive Layers on the Optical Information Transfer Fidelity
The depth advantages that deeper diffractive architectures possess include better generalization capacity for all-optical inference tasks, which has been supported in the literature by both theoretical and empirical evidence.29,31,33
Figure 5.Additional trainable diffractive layers (increasing
2.3 Influence of Limited Phase Bit Depth
In the analyses presented above, we did not impose a bit depth restriction during the training and testing stages, and the encoded phase patterns and the corresponding diffractive heights on each layer were set with a 16-bit depth. In certain applications, the available bit depth of the encoded phase pattern would be limited due to, e.g., the resolution of the SLM; similarly, the bit depth of the diffractive features will be constrained by the 3D fabrication resolution. Motivated to mitigate these challenges, we quantized the electronic encoder and the diffractive decoder that were trained with 16 bits into lower quantization levels in the test stage and investigated the influence of the limited bit depths of the encoded phases’ patterns and the diffractive layers on the quality of the optical information transfer through random new diffusers. Figure 6(a) reports the results of this comparative analysis for several combinations of encoded phase patterns and diffractive decoders with different bit depths. The unconstrained electronic-optical hybrid model’s output results are shown at the top for reference. In this comparison, we first kept the bit depth of the encoded phase patterns as 16-bit while quantizing the diffractive decoder trained with 16-bit to 2-bit, 3-bit, or 4-bit—performing a form of ablation study. The output images of three MNIST handwritten digits and a grating-like object are displayed in Fig. 6(a), revealing a quick improvement in the information transfer performance with increasing diffractive decoder bit depth. We utilized the diffractive decoder (trained under 16-bit depth) using merely 2 bits, and the output image was blurry; however, the output of the 3-bit diffractive decoder improved significantly and the gratings were resolvable [see Fig. 6(a)]. As we further increased the bit depth to 4, the output performance became approximately identical to that of the same model without any bit-depth constraints, as shown in Fig. 6(a).
Figure 6.The impact of the phase bit depth on the optical information transfer fidelity. (a) The joint model trained without any bit-depth limitations, i.e., 16-bit phase representation, and tested with restricted bit-depth phase patterns and decoder pairs. The average PCC value for 10,000 handwritten test digits transmitted through
On the other hand, when we limited the bit depth of the encoded phase patterns while maintaining the diffractive decoder at 16 bits, it was surprising to find out that even with only 1 bit, i.e., binary-encoded phase patterns, the encoder–decoder system remained capable of transferring the input images of interest through unknown random phase diffusers with acceptable output performance [Fig. 6(a)]. Furthermore, the results obtained using the 2-bit phase encoder were comparable to the unconstrained model [Fig. 6(a), top]. These ablation studies highlight the strong robustness of our information encoding strategy to the changes in the phase bit depth that is available. Figure 6(b) reports additional results for various combinations of quantized encoded phase and quantized diffractive decoder, showing that a larger phase bit depth generally leads to better output image performance.
These earlier results were ablation studies performed after the joint training of the electronic-optical model by reducing the available bit depth at the testing phase of the encoder–decoder pair. To explore the minimum requirement of bit depth for accurately transferring optical information through random diffusers, next we adopted the bit-depth limitation during the training process and created three additional electronic-optical models with different levels of bit depth available; see Fig. 7. When both the encoded phase patterns and the diffractive optical decoder were constrained to only 2 bits, the transferred information suffered from significant distortions at the output; however, the contours of some images were still visible. Keeping a bit depth of 2 for the phase encoder while increasing the diffractive decoder’s bit-depth to 4 achieved much better information transfer, where the spatial details of the input images were successfully reconstructed after passing through random phase diffusers. For the combination where both the electronic phase encoder and the optical diffractive models had 4 bits of phase, the performance of optical information transfer through random diffusers was comparable to the hybrid model trained and tested with 16 bits (see Fig. 7).
Figure 7.The electronic encoder and the diffractive decoder trained and tested with different levels of phase bit-depth. The average PCC values are listed for each case.
2.4 Experimental Demonstration of Optical Information Transfer through Unknown Random Phase Diffusers Using a Diffractive Decoder with Electronic Encoding
The presented hybrid electronic-optical model was demonstrated experimentally based on a terahertz continuous-wave (CW) system, operating at
Figure 8.Experimental demonstration of optical information transfer through an unknown random diffuser using a jointly trained pair of an electronic encoder and an optical decoder. (a) Schematic of the joint model for experimental demonstration. (b) Left: height profiles of a new random diffuser (never seen before) and the trained diffractive layers of the all-optical decoder. Right: the fabricated new random diffuser and the diffractive layers used in the experiment. (c) Schematic of the terahertz system. (d) Photograph of the experimental setup and the 3D-printed diffractive decoder.
Figure 9.Experimental results of optical information transfer through an unknown random phase diffuser using the 3D-printed diffractive decoder with electronic encoding.
3 Discussion
We presented a joint electronic-encoder and optical-decoder model designed to transfer optical information through random unknown phase diffusers, outperforming (1) an ideal (diffraction-limited) imaging system and (2) a system solely employing trainable diffractive surfaces, as demonstrated in our previous work.30 With an electronic-encoder CNN encoding the original input images into 2- to 4-bit depth phase patterns, a jointly trained diffractive optical decoder becomes much more resilient to the distortions caused by random, unknown phase diffusers along the optical path, leading to enhanced performance and higher data transmission fidelity. The diffractive optical decoder, consisting only of passive diffractive layers, can decode the encoded information through random phase diffusers at the speed of light and enable the hybrid model to work with low power consumption. The overall volume of the diffractive optical decoder is also compact, with an axial span of
Note that the phase encoding strategy employed in our models could also be interchanged with an amplitude encoding scheme, while sustaining a comparable level of performance and data transmission fidelity across random, unknown diffusers. To showcase this, we trained an alternative hybrid electronic-optical model using amplitude encoding to transfer optical information through random phase diffusers with a correlation length of
In our results reported so far, the positioning of the diffusers remained constant during the training and testing phases. To assess the impact of unknown diffuser location on the data transmission fidelity, we blindly tested the model shown in Fig. 1(a), trained using a fixed diffuser location, with varying axial positions spanning
Following the previous literature on random phase diffusers,21,22,39,40 in this work, we used thin optical elements with random phase patterns to simulate diffusers; the exploration of volumetric diffusers, which can be described by several thin phase diffusers and split-step beam propagation, is an exciting future research direction.41,42 In addition, wavelength-division multiplexing, widely used in fiber-optic communication to enable high transmission bandwidths,43 can be integrated into the presented jointly trained models, allowing simultaneous information transfer at multiple wavelengths and increasing the overall capacity of information transfer through random unknown diffusers. Finally, our method can be physically scaled (expanded/shrunk) with respect to the illumination wavelength without retraining its components and can operate at different parts of the electromagnetic spectrum to transfer optical information through random scattering media.
4 Appendix
4.1 Electronic Encoder Design
We used a CNN to encode the input object into a 2D phase pattern. The network architecture is shown in Fig. S1 in the Supplementary Material, which has convolutional layers with
4.2 All-Optical Diffractive Decoder Model
The monochrome illumination with a wavelength of
Both the amplitude modulation
We modeled the random diffusers as pure phase elements, whose complex transmission
Adjacent optical elements (e.g., input phase patterns, random diffusers, diffractive layers, and detectors) are optically connected by free-space light propagation in air, which was formulated using the Rayleigh–Sommerfeld equation.46 The propagation can be modeled as a shift-invariant linear system with the impulse response,
Considering a plane wave that is incident at a phase-modulated object
This distorted field is used as the input field of subsequent diffractive layers. And the optical field
4.3 Digital Implementation
For the diffractive decoder, we sampled the 2D space with a grid of
During the joint training process of the electronic encoder and the diffractive decoder using deep learning, we first normalized the handwritten digits from the MNIST training data set to the range [0, 1] and bilinearly interpolated them from
During the training,
To validate the necessity of employing a hybrid electronic-encoder and optical-decoder model for optical information transmission through random phase diffusers, we trained both the hybrid model and the diffractive decoder independently using various values of
The electronic encoder and the diffractive optical decoder were jointly trained using Python (v3.8.16) and PyTorch (v1.11, Meta AI) with a GeForce RTX 3090 graphical processing unit (Nvidia Corp.), an Intel® Core™ i9-12900KF central processing unit (Intel Corp.), and 64 GB of RAM, running the Windows 11 operating system (Microsoft Corp.). The calculated loss values were backpropagated to update the weights and biases of the electronic encoder and diffractive neuron heights using the Adam optimizer47 with a decaying learning rate of
4.4 Quantization of the Encoded Phase and the Diffractive Decoder Layers
To evaluate the performance of the hybrid electronic-encoder and optical-decoder models under limited bit-depth cases, we quantized the encoded phase and/or the heights of the features on the diffractive layers into lower quantization levels. For the encoded phase patterns with a maximum phase of
4.5 Terahertz Experimental Setup and Design
The experimental setup is shown in Fig. 8(b). The terahertz source used in the experiment was an AMC (modular amplifier, WR9.0M SGX, Virginia Diode Inc.; multiplier chain, WR4.3x2 WR2.2x2, Virginia Diode Inc.) with a compatible diagonal horn antenna (WR2.2, Virginia Diode Inc.). The input of the AMC was a 10 dBm RF input signal at 11.1111 GHz (
The phase objects, diffusers, and diffractive layers used for experimental demonstration were fabricated using a 3D printer (Objet30 Pro, Stratasys). The absorption of the 3D printing material was taken into account and estimated by Eq. (2) for the phase patterns, diffusers, and diffractive layers during the training. For the two-layer decoder model used in the experimental demonstration, each diffractive layer consisted of
To build the resilience of the diffractive decoder layers to potential mechanical misalignments in the experimental testing, the diffractive decoder was “vaccinated” with deliberate random shifts during the training.38 For this vaccination process, a random lateral displacement (
A random axial displacement of
Considering the material absorption and minimum detectable signal in our experiments, we also added another power penalty to balance the output quality and the diffraction efficiency. We calculated the power efficiency
4.6 Image Contrast Enhancement
To better visualize the images, we digitally enhanced the contrast of the experimental measurements using a built-in MATLAB function (imadjust), which by default saturates the top 1% and the bottom 1% of the pixel values and maps the resulting images to a dynamic range between 0 and 1. All the quantitative data analyses, including PCC calculations and resolution test target results, are based on raw data without applying image contrast enhancement.
4.7 Simulation of the Standard Lens-Based Image Transmission System
We numerically implemented a conventional lens-based image transmission system to evaluate the impact of a given random diffuser on the output image. A Fresnel lens was designed to have a focal length (
The lens was placed
Yuhang Li received his BS degree in optical science and engineering from Zhejiang University, Hangzhou, China, in 2021. He is currently working toward his PhD in Electrical and Computer Department at the University of California, Los Angeles, California, United States. His work focuses on the development of computational imaging, machine learning, and optics.
Tianyi Gan received his BS degree in physics from Peking University in 2021. He is currently a PhD student in the Electrical and Computer Engineering Department at the University of California, Los Angeles. His research interests are terahertz source and imaging.
Bijie Bai received her BS degree in measurement, control technology and instrumentation from Tsinghua University, Beijing, China, in 2018. She is currently working toward her PhD in the Electrical and Computer Engineering Department, University of California, Los Angeles, California, United States. Her research focuses on computational imaging for biomedical applications, machine learning, and optics.
Çağatay Işıl graduated with a BS degree in electrical-electronics engineering (EE) in 2017 and a BS degree in physics in 2018 from Middle East Technical University in Ankara, Turkey. Subsequently, he completed his MS degree in EE at the same institution. Currently, he is actively engaged in pursuing his doctorate in the Electrical and Computer Engineering Department at the University of California, Los Angeles (UCLA), located in California, United States. His research is primarily centered around the advancement of virtual staining techniques and all-optical machine-learning systems.
Mona Jarrahi is a professor and Northrop Grumman Endowed Chair in electrical and computer engineering at UCLA and the director of Terahertz Electronics Laboratory. She has made significant contributions to the development of ultrafast electronic and optoelectronic devices and integrated systems for terahertz, infrared, and millimeter-wave sensing, imaging, computing, and communication systems by utilizing novel materials, nanostructures, and quantum structures, as well as innovative plasmonic and optical concepts.
Aydogan Ozcan is the Chancellor’s Professor and the Volgenau Chair for Engineering Innovation at UCLA and an HHMI professor with the Howard Hughes Medical Institute. He is also the associate director of the California NanoSystems Institute. He is elected fellow of the National Academy of Inventors (NAI) and holds more than 65 issued/granted patents in microscopy, holography, computational imaging, sensing, mobile diagnostics, nonlinear optics, and fiber-optics, and is also the author of one book and the co-author of more than 1000 peer-reviewed publications in leading scientific journals/conferences. He is an elected fellow of Optica, AAAS, SPIE, IEEE, AIMBE, RSC, APS, and the Guggenheim Foundation, and is a lifetime fellow member of Optica, NAI, AAAS, and SPIE. He is also listed as a highly cited researcher by Web of Science, Clarivate.
[31] M. S. S. Rahman et al. Universal linear intensity transformations using spatially-incoherent diffractive processors(2023).
[44] S. Ioffe, C. Szegedy. Batch normalization: accelerating deep network training by reducing internal covariate shift, 448-456(2015).
[46] J. W. Goodman. Introduction to Fourier Optics(2005).
[47] D. P. Kingma, J. Ba. Adam: a method for stochastic optimization(2015).
Get Citation
Copy Citation Text
Yuhang Li, Tianyi Gan, Bijie Bai, Çağatay Işıl, Mona Jarrahi, Aydogan Ozcan, "Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding," Adv. Photon. 5, 046009 (2023)
Category: Research Articles
Received: Mar. 30, 2023
Accepted: Jul. 13, 2023
Posted: Jul. 13, 2023
Published Online: Aug. 30, 2023
The Author Email: Ozcan Aydogan (ozcan@ucla.edu)