Snapshot computational spectroscopy enabled by deep learning
Sep. 07 , 2024photonics1

Abstract

Spectroscopy is a technique that analyzes the interaction between matter and light as a function of wavelength. It is the most convenient method for obtaining qualitative and quantitative information about an unknown sample with reasonable accuracy. However, traditional spectroscopy is reliant on bulky and expensive spectrometers, while emerging applications of portable, low-cost and lightweight sensing and imaging necessitate the development of miniaturized spectrometers. In this study, we have developed a computational spectroscopy method that can provide single-shot operation, sub-nanometer spectral resolution, and direct materials characterization. This method is enabled by a metasurface integrated computational spectrometer and deep learning algorithms. The identification of critical parameters of optical cavities and chemical solutions is demonstrated through the application of the method, with an average spectral reconstruction accuracy of 0.4?nm and an actual measurement error of 0.32?nm. The mean square errors for the characterization of cavity length and solution concentration are 0.53?% and 1.21?%, respectively. Consequently, computational spectroscopy can achieve the same level of spectral accuracy as traditional spectroscopy while providing convenient, rapid material characterization in a variety of scenarios.

1 Introduction

In numerous scientific disciplines, spectroscopic analysis represents a fundamental and powerful tool that utilizes the absorption, emission, or scattering properties of materials in response to electromagnetic radiation (light) to reveal information about their composition and structure [1], [2]. This instrumental technique plays a pivotal role in the characterization and analysis of materials science, food science, nanotechnology, biology and chemistry, among other fields. For instance, the refractive index of thin film materials can be quantified by ellipsometry, while the characteristic absorption peaks of optical fiber materials and their quality can be readily identified by absorption spectroscopy [3]. Fourier transform infrared spectrometers are frequently employed to identify the chemical composition and solution concentration of chemical and biological samples that are commonly encountered in the food, environmental and medical industries, where characteristic absorbing and extinctive spectral features are studied [4]. Surface-enhanced Raman spectroscopy (SERS) is a highly sensitive spectroscopic method that enhances the Raman scattering of molecules and allows for the structural fingerprinting of low concentration analytes through the plasmon-mediated amplification of electric fields or chemical enhancement [5]. The extraordinary spectral resonance of plasmonics and the dielectric metasurface can be readily investigated through light transmission/reflection/scattering measurements [6]. The spectral information encodes a wealth of information not only about the material composition, but also the critical parameters of the photonic structure and cavity configuration, among other things. Consequently, it is of paramount importance to develop accurate and accessible spectroscopic techniques.

The recent rapid development and wide application of deep learning tools has enabled the direct inverse design and optimisation of photonic devices using a large amount of materials (or structures) and their corresponding spectral data [7], [8]. A deep learning approach can analyze the scattering spectra of silicon nanostructures within the diffraction-limited region and output digital information, which breaks the limits of optical information storage and already surpasses the blu-ray disc technology [9]. Recently, a deep neural network has also been employed to characterize the critical geometric and material parameters of unknown plasmonic nanostructures through simple transmission spectra [10], suggesting a simple and intelligent approach to material characterization using spectroscopic techniques.

Neverthless, high-quality spectroscopic data is typically obtained using conventional spectrometers. Conventional spectrometers typically comprise monochromators and Fourier transform spectrometers. Monochromators typically employ prisms or gratings to divide the light source into monochromatic light [11]. This process is conducted sequentially, necessitating a sufficiently long optical path to separate different wavelengths and achieve high spectral resolutions. Fourier transform spectrometers use the Michelson interferometer, which generates an interference pattern by creating a path difference between two light beams using movable mirrors [12]. The long optical length and bulky dispersive elements in these conventional spectrometers impede the adoption of low-cost, portable, fast and accurate spectroscopic analysis in everyday scenarios.

In contrast to conventional geometric optics, nanophotonic structures can extend the length of the optical path to millions of times larger than their physical dimensions. Nanophotonic structures themselves exhibit a distinctive light–matter interaction, known as the response function. Recently, with the assistance of appropriate computational algorithms based on compressive sensing theory, the spectral information can be accurately decoded and reconstructed by a multitude of miniature spectrometers [13], [14], [15]. The integration of photonic crystal structures or metasurfaces with complementary metal oxide semiconductor (CMOS) or charge-coupled device (CCD) sensors has led to the development of compact and low-cost spectral analysis tools [16], [17]. These innovations use interference patterns generated within photonic structures to encode and decode spectral information [15], [18]–[21]. Furthermore, computational spectrometers can reconstruct high-resolution spectral images from low-resolution images or sparse sampling data through deep learning, which effectively compresses spectral data, removes noise, and uses parallel computing capabilities to process large amounts of spectral data in real-time [22]–[25]. These advances facilitate the development of spectrometers with minimal signal correlation, which significantly improves the accuracy of spectral reconstruction and analysis. In addition to nanophotonic integrated image sensors, research on semiconductor nanowires has also achieved dynamic control of the photon response, making significant contributions to the miniaturization of spectroscopic instruments [26]. If the spectroscopic data can be collected by computational spectrometers, structural and material analysis can be easily decoded by deep learning algorithms, enabling a snapshot type of portable, low-cost computational characterization tool. Meng’s work has demonstrated that machine learning combined with filter arrays can be applied to material classification in theoretical simulations [27].

This work presents a novel snapshot computational spectroscopy (SCS) tool consisting of a metasurface integrated computational spectrometer (MICS) and a deep neural network (DNN), which is capable of rapid material analysis. The principle of operation is illustrated in Figure 1. The structural and material information of the sample of interest is contained in the transmission/reflection spectra, which are then captured by a snapshot light intensity image taken by the metasurface integrated CCD imager. The spectral information is then decoded by the reconstruction algorithm and fed into a deep neural network for direct classification and regression. All computationally reconstructed spectra achieve comparable spectral detection accuracies to those of traditional spectrometers, while the overall size of the computational spectroscopic tool is only approximately 100?mm × 50?mm × 50?mm, with a spectral sampling speed of less than 1?s. This approach provides a novel theoretical framework for the rapid and accurate prediction of material properties, while simultaneously addressing the need for portability, miniaturization, accuracy and speed of spectral acquisition and analysis.

Figure 1: Schematics of snapshot computational spectroscopy enabled (SCS) by metasurface integrated computational spectrometer (MICS) and deep neural network (DNN). The white light source irradiates the material to be tested to form spectral information, which is then encoded by an encoder and recorded by CCD image sensor. The spectral information is then decoded by reconstruction algorithm, and finally fed into a DNN for direct characterization.
Figure 1:

Schematics of snapshot computational spectroscopy enabled (SCS) by metasurface integrated computational spectrometer (MICS) and deep neural network (DNN). The white light source irradiates the material to be tested to form spectral information, which is then encoded by an encoder and recorded by CCD image sensor. The spectral information is then decoded by reconstruction algorithm, and finally fed into a DNN for direct characterization.

2 Methods

2.1 The design and fabrication of metasurface filter array

The accuracy of spectral reconstruction is related to the size and correlation of metasurface arrays as previously discussed [28]. For better spatial resolution required in spectral imaging, a small filter array is desirable; however, this necessitates a low response function correlation for accurate spectral reconstruction. Conversely, a large filter array encodes spectra with sufficient data points, and the reconstruction accuracy is almost unaffected by the array correlation coefficient if the array size is greater than 7 × 7 structures from previous investigations [28], [29], [30]. In our work, in order to achieve a high level of spectral reconstruction accuracy, we selected a 10 × 10 large array structure as an encoder from the filter design database based on our previous inverse design approach. Afterwards, a thin layer of UV glue is spin coated on the surface of the CCD sensor to bond the processed metasurface array chip (with sapphire substrate) to the sensor surface after ultraviolet radiation. The metasurface array chip can thus be integrated onto the CCD sensor as illustrated in Figure 2(a) and (b).

Figure 2: The metasurface array structures and their response functions. (a) Photograph of metasurface array mounted on a CCD image sensor. (b) An image of the 100 metasurface filter array under a microscope. (c) Scanning electron microscope of representative metasurfaces, including structures with different shapes of crosses, circles, squares, and varied diameters, periodicities, and angles between rows. (d) The simulated transmission spectra corresponding to the six structures shown in Figure c and the transmission spectra measured by MStarter ABS micro area spectrometer. (e) The correlation matrix of the overall transmission spectra of the 100 metasurface filter array.
Figure 2:

The metasurface array structures and their response functions. (a) Photograph of metasurface array mounted on a CCD image sensor. (b) An image of the 100 metasurface filter array under a microscope. (c) Scanning electron microscope of representative metasurfaces, including structures with different shapes of crosses, circles, squares, and varied diameters, periodicities, and angles between rows. (d) The simulated transmission spectra corresponding to the six structures shown in Figure c and the transmission spectra measured by MStarter ABS micro area spectrometer. (e) The correlation matrix of the overall transmission spectra of the 100 metasurface filter array.

In order to optimize the array design, the Lumerical FDTD Solutions software was employed to obtain the filter design database, which contains five hundred metasurfaces with different transmittances (response functions) resulting from different shapes, sizes, orientations and periods of the meta-atoms. This is illustrated in Figure 2(c). This metasurface database is relatively small as we need to consider the nanofacrication feasibility and effective response in the visible domain. In order to select the optimized array structure with the least correlation in this relatively small database, the parameterized design of the metasurface was fed into a deep neural network (DNN) along with the corresponding transmission spectra, as show in Figure S1(a). The loss function was defined to minimize the discrepency between the model output and target output, including a regularization term to assess the correlation of different structural groups. The transmission spectra of all designs are computed by forward propagation through the network, and an algorithm is used to adjust the parameters of the metasurface structures to minimize the loss function. Through this method, we are able to select 100 filter designs with the lowest correlation from the validation results and the final array exhibits an average correlation value of 0.681 which is sufficient for accurate spectrum reconstruction [28]. In addition, the actual correlation value deviates from the design a bit due to unavoidable nanofabrication error induced spectral response discrepancy, our simulation correlation is 0.57.

Following the selection the structural parameters, the sapphire substrate was coated with a 100?nm thick silicon film through plasma-enhanced chemical vapor deposition (PECVD), which was identical to the simulated conditions. Each fabricated metasurface filter in the array is approximately 15?μm in size, with a 2?μm spacing between each other, resulting in a total array size of 170?μm × 170?μm. The metasurface patterns are transferred to the silicon film by electron beam lithography (EBL) followed by reactive ion etching (RIE) to obtain the ultimate metasurface filter array chip. Subsequently, the transmission spectra of each metasurface in the fabricated encoder chip was measured by using MStarter ABS with a spot size of 5?μm and a wavelength range from 400?nm to 900?nm. Figure 2(d) illustrates the simulated and experimental response functions for six representative metasurface structures. There is unavoidable spectral response discrepancy between the simulation and fabrication due to unavoidable nanofabrication errors. All the 100 transmissive response functions are converted into a response matrix equation T i (λ), and the actual measured array correlation is calculated and presented in Figure 2(e).

2.2 The calibration of computational spectrometer and spectral reconstruction

In the construction of a miniaturized computational spectrometer, a white LED point light source is employed for the purpose of system miniaturization. In order to convert the scattered light from the point source into uniform, parallel light, a thin convex lens is installed behind the light source. The thin convex lens is capable of shaping the light into parallel beams, which are necessary for subsequent CCD signal collection. Given that there is always a photon loss during the photosensing process, it is of the utmost importance to calibrate CCD light intensity sensitivity, denoted by η(λ) prior to reconstruction. The calibration ultilizes a monochromator with its optical path illustrated in Figure S2(a). The operational mode may be either reflection or transmission, as illustrated in Figure S2(b) and (c). The CCD industrial camera model we use is MV-CU120-10?GM, with a single pixel size of 1.85?μm × 1.85?μm. The response curve as shown in Figure S3. The composition or structure of the sample may either absorb or scatter light, resulting in a unique spectral fingerprint. Subsequently, the spectral information is collected by the CCD image and converted into an electrical signal for subsequent decoding processing, following its entry into the metasurface array encoder. For calibration purposes, the monochromator was employed to sequentially scan the wavelength range from 400?nm to 900?nm with a 0.4?nm bandwidth. At each wavelength, the light intensity signals were collected using a CCD and a photometer. The incorporation of the light intensity sensitivity η(λ) of the CCD image sensor, calculated and incorporated into the reconstructed spectrum, can result in a more realistic and accurate signals, thereby improving the accuracy of spectral reconstruction. Finally, the incident spectrum I(λ) can be reconstructed using the collected CCD image sensor’s electrical signals S i , the response matrix equation T i (λ), and the spectral sensitivity η(λ) through the spectral reconstruction formula (1), as previously described in numerous works [14], [16], [17], [31]. The detailed principle can be obtained in Figure S4.

(1) Si=Tiλ*ηλ*Iλ

2.3 The training of deep neural network for spectral characterization

Furthermore, in order to perform computational characterization with high accuracy and efficiency, a DNN network was constructed. The DNN comprises of a 1 × 1,251 input layer for the reconstructed spectra from 400 to 900?nm, 3 × 2,000 fully connected hidden layers, a 1 × n regression network output layer, and a 1 × m classification network output layer (where n is the number of labels required for the regression network output, and m is the number of labels for the classification network output). The activation function of the regression network is the rectified linear unit (ReLU), while that of the classification network is the Softmax function. The DNN neural network architecture is shown in Figure S1(b). As demonstrated in our previous research, experimental and simulated spectra exhibit common resonance features. An accurate experimental characterization of the material property was demonstrated using DNN training on mixed experimental data and simulated data [10]. Learning from this, we have used only simulated data for DNN training and the test dataset consists only of experimental data. In this way, the burden of collecting sufficient experimental data as a training dataset is greatly reduced.

3 Results

3.1 The spectral reconstruction accuracy analysis

In order to verify the precision and accuracy of our miniaturized computational spectrometer, we collected spectral information from each sample using both high-precision commercial UV–Vis–NIR spectrometer (Agilent Cary5000) and our MICS approach. First, we validated the reconstruction accuracy of simple spectra from narrow pulse spectra in the 400?nm–900?nm wavelength range. The narrow pulse spectra were obtained over a wavelength range of 450?nm–850?nm with a 50?nm pulse interval and a spectral full width at half maximum (FWHM) of 4?nm, as shown in Figure 3(a). The dashed lines represent the transmission spectra measured by the Cary5000, while the solid lines represent the transmission spectra reconstructed by MICS. An enlarged spectral peak comparison of a 650?nm pulse signal is shown on the right. The reconstructed spectra of incandescent and LED light sources have high reconstruction accuracy, as shown in Figure 3(b) and (c). We also reconstructed a narrow spectrum with a peak shift difference of only 0.4?nm, demonstrating that our spectral reconstruction accuracy reached 0.4?nm, as shown in Figure S5.

Figure 3: Spectral reconstruction accuracy analysis. (a) The left image shows the spectra of pulse light with a full width at half maximum (FWHM) of 4?nm and intervals of 50?nm in the 400?nm–900?nm range, detected by both traditional spectrometers and miniature computational spectrometers. The dashed lines represent the results from Agilent Cary5000, while the solid lines are the results from MICS. The right image shows the enlarged spectral detection results for the pulse spectrum centered at 650?nm. (b) And (c) measured and reconstructed spectra of two different light sources. (d) Transmission spectra of a metasurface before and after spin-coating of an optical adhesive NOA 63, detected by both types of spectrometers. (e) The reflection spectra of two optical metasurfaces with different periods and diameters. (f) The transmission spectra of the methyl blue solution concentration with a difference of 0.01?g/L. (g) The transmission spectra of metal-dielectric-metal multilayer film with different dielectric layer thicknesses. (h)–(k) Mean squared error of spectrum, average peak shift error, average amplitude error and FWHM error in (a), (d)–(g).
Figure 3:

Spectral reconstruction accuracy analysis. (a) The left image shows the spectra of pulse light with a full width at half maximum (FWHM) of 4?nm and intervals of 50?nm in the 400?nm–900?nm range, detected by both traditional spectrometers and miniature computational spectrometers. The dashed lines represent the results from Agilent Cary5000, while the solid lines are the results from MICS. The right image shows the enlarged spectral detection results for the pulse spectrum centered at 650?nm. (b) And (c) measured and reconstructed spectra of two different light sources. (d) Transmission spectra of a metasurface before and after spin-coating of an optical adhesive NOA 63, detected by both types of spectrometers. (e) The reflection spectra of two optical metasurfaces with different periods and diameters. (f) The transmission spectra of the methyl blue solution concentration with a difference of 0.01?g/L. (g) The transmission spectra of metal-dielectric-metal multilayer film with different dielectric layer thicknesses. (h)–(k) Mean squared error of spectrum, average peak shift error, average amplitude error and FWHM error in (a), (d)–(g).

In order to validate the spectral reconstruction accuracy and resolution for actual nanophotonic structures and chemical solutions, a number of cases have been selected for test and comparison. The first group comprises two metasurfaces comprising circular nanoposts with a period of 1,300?nm and a diameter of 700?nm. One of these metasurfaces has been spin coated with a thin layer of NOA 63 with a refractive index value of 1.56, situated above the metasurface (Figure S6(a)). The transmission spectra of each sample were measured and are shown in Figure 3(d). The dashed line represents the measurement by the Cary5000, while the solid line represents the measurement by MICS. The second group comprises circular nanopost metasurfaces with the following parameters: Period = 400?nm, Diameter = 300?nm, and P = 300?nm, D = 200?nm (Figure S6(b)). Their transmission spectra are shown in Figure 3(e). The third group is composed of methyl blue solutions with a concentration difference of 0.01?g/L (Figure S6(c)), and its transmission spectrum is shown in Figure 3(f). The fourth group is composed of a simple metal-dielectric-metal multilayer film configuration, where the top metal layer is Ag, the bottom layer is Au, and the insulator layer is methyl methacrylate (MMA). The dielectric thickness of the insulator layer is 280?nm and 340?nm, respectively (Figure S6(d)). Their transmission spectra are shown in Figure 3(g). A direct comparison of the reconstructed and measured spectra reveals that the computational spectrometer is capable of accurately reproducing the spectral characteristics of the measured data.

A quantitative analysis was conducted on four aspects of the above spectra, including the mean square error (MSE), peak shift error, amplitude error and full width at half maximum (FWHM) error in reconstructed spectra. The results presented in Figure 3(h) – (k) demonstrate that the average MSE for all spectral reconstructions was 1.28 × 10−4, the average peak shift error for all reconstructed spectra was 0.32?nm, and the average peak amplitude error was 0.6?%, the FWHM error is 0.437?nm. The analysis revealed that the reconstruction offset error of a miniaturised computational spectrometer was 0.32?nm, which was smaller than its spectral resolution of 0.4?nm. Consequently, the reconstruction accuracy of MICS is 0.4?nm, with a reconstruction accuracy of 99.4?%, closely matching the spectra detected by traditional spectrometers. Within the current study range of 400?nm–900?nm, the MICS can be employed as a substitute for traditional spectrometers for spectral analysis.

3.2 Material and structural computational characterization

The application of the SCS technique to the characterisation of a simple optical cavity is demonstrated, based on the highly accurate spectral information sensed by the MICS technique. The Fabry–Pérot (FP) cavity is a common optical cavity employed in a variety of laser, spectrometer, and optical instrument applications. An FP cavity is composed of two parallel mirrors and an adjustable cavity length. This phenomenon of wavelength-dependent interference can either enhance or diminish the transmission spectrum, resulting in the formation of peaks. The characterisation of the FP cavity length typically relies on sophisticated tools such as scanning or tunneling electron microscopes for cross-sectional examination. The precise relationship between the cavity length and the peak wavelength can be expressed as 2nd = , where n is the refractive index within the cavity, d is the cavity thickness, m is an integer, and λ is the wavelength of the transmitted light. If the total path length is an integer multiple of the wavelength of the light, the reflected light waves will constructively interfere, forming interference peaks.

In accordance with the constructive interference condition, as the cavity length of the FP cavity increases, the wavelength supporting constructive interference shifts to longer wavelengths, as illustrated in Figure 4(b). Consequently, the identification of the peak positions of the transmission spectra allows for the characterisation of the cavity length. As deep neural networks (DNNs) can map the physical relationship between structure and spectra with high accuracy, we have incorporated a DNN to analyse the reconstructed spectra, with the aim of predicting the material type, cavity length and cavity configuration, among other things, in an intelligent manner. The initial training set comprised three-layer FP cavities simulated using Lumerical FDTD Solutions, with varying thicknesses and materials. The training set included two metals (Au, Ag) and three dielectrics (PMMA, MMA, Al2O3) with dielectric layer thicknesses ranging from 300?nm to 900?nm, at 10?nm intervals, totalling 26,000 datasets. The degree of fit between the simulated and experimental spectra was found to be between 91.2?% and 98.0?% after data normalization and preprocessing, as illustrated in Figure S7S9. The entire set of simulated data was employed as the training set for the DNN network, with the trained DNN subsequently used for the prediction of experimental spectra.

Figure 4: The application of snapshot computational spectroscopy. (a) Schematic diagram of the Fabry–Pérot (FP) cavity. (b) Transmission spectra of FP cavities with different dielectric cavity lengths measured by Carry5000. (c) Transmission spectra of FP cavities with different dielectric cavity lengths measured by MICS. (d) Schematic diagram of the solution samples with different concentrations. (e) Transmission spectra of methyl blue solutions with different concentrations measured by Carry5000. (f) Transmission spectra of methyl blue solutions with different concentrations measured by MICS. (g) Loss curve after the DNN network training is complete. (h) Changes in the success rate of prediction for the regression and classification networks during the hyperparameter optimization process. (i) Statistics of the mean squared error between the predictions from data obtained from Cary5000 and MICS. (j) Confusion matrix for predicting solute types using classification networks.
Figure 4:

The application of snapshot computational spectroscopy. (a) Schematic diagram of the Fabry–Pérot (FP) cavity. (b) Transmission spectra of FP cavities with different dielectric cavity lengths measured by Carry5000. (c) Transmission spectra of FP cavities with different dielectric cavity lengths measured by MICS. (d) Schematic diagram of the solution samples with different concentrations. (e) Transmission spectra of methyl blue solutions with different concentrations measured by Carry5000. (f) Transmission spectra of methyl blue solutions with different concentrations measured by MICS. (g) Loss curve after the DNN network training is complete. (h) Changes in the success rate of prediction for the regression and classification networks during the hyperparameter optimization process. (i) Statistics of the mean squared error between the predictions from data obtained from Cary5000 and MICS. (j) Confusion matrix for predicting solute types using classification networks.

Subsequently, 100 sets of FP cavities with different metals, dielectrics, and thicknesses were measured using MICS and Cary 5000, which served as the test set for the pre-trained DNN network to analyze its prediction success rate. The DNN network has two outputs: one is the classification network output, which is designed to predict the type of metal and dielectric material present in the FP cavity, and the other is the regression network output, which is intended to predict the thickness of the FP cavity dielectric layer. The prediction performance for different dielectric layer thicknesses is illustrated in Figure S10(g) and (h). The dielectric layer of aluminum oxide exhibited the highest MSE, with a prediction error of 0.76?nm by Cary 5000 and 0.81?nm for MICS. The lowest mean square error (MSE) was observed for MMA, with a prediction error of 0.23?nm for Cary 5000 and 0.25?nm for MICS, respectively. Both types of spectrometers achieved a 100?% success rate in predicting the type of materials, as demonstrated in Figure S10(a)–(f), which present the confusion matrices for the predictions made by the two spectroscopic data sets.

Another typical application example is the spectroscopic analysis of chemical solutions. When light passes through a solution, a portion of the light is absorbed by the solute present within the solution. The quantity of light absorbed is contingent upon the nature and concentration of the solute, and this relationship is elucidated by Beer’s Law. The law states that the absorption of light at a specific wavelength is directly proportional to the concentration of the solute, with the formula A = ? × c × l, where A is the absorbance, ? is the molar absorptivity, c is the molar concentration of the solute, and l is the path length of the light through the solution. Transmission spectra demonstrate the intensity of light that has traversed the solution, reflecting the quantity of light that has not been absorbed. Figure 4(e) illustrates the manner in which the transmission spectrum of a methylene blue solution varies with concentration. Given that different solutions exhibit varying absorption coefficients, transmission spectra can be employed to ascertain the nature and concentration of solutions. In the absence of knowledge regarding the concentration of the solution, it is necessary to characterise the solution’s properties using a range of common methods, including spectrophotometry, titration, conductivity determination and nuclear magnetic resonance spectroscopy, among others.

In order to illustrate the SCS technique in chemical solution characterisation, four types of solutions were selected, each containing a different chemical: methylene blue (Bs), rhodamine (AR), Cu2+ ions and Fe3+ ions. The concentrations of Bs and AR ranged from 0.01?g/L to 1?g/L, while those of Cu2+ and Fe3+ ranged from 1?g/L to 100?g/L, resulting in a total of 80 datasets. All the measured experimental spectra were used for training purposes. In order to expand the dataset size, random errors were added for data augmentation, resulting in a total of 720 datasets. Of these, 80?% were selected for training, while the remaining 20?% were used for testing. As the selection of hyperparameters can directly impact the performance and efficiency of models in deep learning and cannot be automatically updated during training, we also incorporated hyperparameter optimisation. The hyperparameters that were optimized included the number of neurons, the activation function, the number of epochs, the batch size, and the number of optimization rounds, which were set at 10. Hyperparameter optimization can markedly enhance model performance, prevent overfitting, accelerate training, and accommodate diverse data characteristics.

Figure 4(i) illustrates the efficacy of the classification and regression networks during hyperparameter optimization. A regression network for the FP cavity is considered successful if the predicted thickness error is less than 1?nm from the actual thickness error. For solutions, a success is defined when the concentration error is less than 2?%. The DNN network loss curve is depicted in Figure 4(h). Upon inputting the test dataset of solutions into the trained DNN network, the prediction results for varying solution concentrations are presented in Figure S11(c) and (d), with an overall concentration mean square error of 1.8?%. The results of solution type predictions by both the Cary 5000 and MICS are shown in Figure S11(a) and (b), respectively, with 100?% accuracy. The overall mean squared error (MSE) for the FP cavities and solutions is displayed in Figure 4(j), which demonstrates that the chemical solution characterisation by both spectrometers exhibits negligible differences in the predicted errors.

4 Discussion

The operational scope of traditional spectrometers is contingent upon the spectral range of gratings and photodetectors, whereas computational spectrometers are also constrained by the dynamic range of encoders and image sensors. The operational range of the computational spectrometer described in this article extends from 400?nm to 900?nm, corresponding to the photosensitive range of the CCD image sensor and the design of the metasurface array encoder. However, this does not preclude the potential applications of this MICS technology in the ultraviolet and infrared bands. Furthermore, in order to expand the application of the SCS technique in reality, it is necessary to collect sufficient spectral data for various materials in order to form pretrained DNN tools. This necessitates the acquisition of a considerable quantity of experimental spectral data and the subsequent training of DNNs.

5 Conclusion

The combination of a MICS hardware and a DNN software tool can form a snapshot computational spectroscopic technique, which can rapidly and accurately identify a sample’s critical properties such as the DNN network and chemical solution concentration. This represents an innovative alternative to traditional spectrometers and provides a new theoretical foundation for the rapid and precise characterisation of materials. In the future, by optimising the design and fabrication of encoders and photodetectors, the sampling range of miniature computational spectrometers can be broadened to the ultraviolet and infrared regions, with the potential for further miniaturisation. The integration of a metasurface with two-dimensional photodetectors enables the realization of an ultraminiature configuration with micrometer dimensions. The technology is compatible with the CMOS technology, which facilitates the integration of MICS within a smartphone. This integration enables the development of truly portable SCS applications for spectral detection and material characterization in a range of biomedical, food and environmental test scenarios.

Supporting Information

Supporting information includes spectral reconstruction algorithm, spectral detection of different substances, feasibility analysis of FP cavity prediction and analysis of prediction results of the DNN networks.


Corresponding authors: Huijuan Zhao, School of Materials Science and Engineering, Nanjing University of Posts and Telecommunications, 9 Wenyuan Road, 210023, Nanjing, China, E-mail: ; and Li Gao, School of Materials Science and Engineering, Nanjing University of Posts and Telecommunications, 9 Wenyuan Road, 210023, Nanjing, China; and School of Science, Nanjing University of Posts and Telecommunications, 9 Wenyuan Road, 210023, Nanjing, China, E-mail:

Funding source: the National Key Research and Development Program of China

Award Identifier / Grant number: 2021YFA1202904

Funding source: National Natural Science Foundation of China

Award Identifier / Grant number: 62288102

Award Identifier / Grant number: 62375139

Funding source: the Project of State Key Laboratory of Organic Electronics and Information Displays

Award Identifier / Grant number: GDX2022010007

Funding source: Jiangsu Province Key R&D Program

Award Identifier / Grant number: No. BK20232044

 
  1. Research funding: The work at Nanjing University of Posts and Telecommunications was supported by the National Key Research and Development Program of China (2021YFA1202904); the National Natural Science Foundation of China (62375139, 62288102); the Project of State Key Laboratory of Organic Electronics and Information Displays (GDX2022010007); and Jiangsu Province Key R&D Program (No. BK20232044).

  2. Author contributions: All authors have accepted responsibility for the entire content of this manuscript and approved its submission.

  3. Conflict of interest: Authors state no conflicts of interest.

  4. Data availability: The data that support the findings of this study are available from the authors upon reasonable request.

References

[1] X. L. Chu, H. F. Yuan, and W. Z. Lu, “Progress and application of spectral data pretreatment and wavelength selection methods in NIR analytical technique,” Prog. Chem., vol. 16, no. 4, pp. 528–542, 2004.Search in Google Scholar

[2] X. C. Shen, H. Liang, X. W. He, and X. S. Wang, “Recent trends and spectroscopic methods for analysis of the protein conformation with circular dichroism,” Chin. J. Anal. Chem., vol. 32, no. 3, pp. 388–394, 2004.Search in Google Scholar

[3] R. Rodríguez-López, N. Abundiz-Cisneros, R. Sanginés, J. Aguila-Muñoz, and R. Machorro-Mejía, “Plasma emission spectroscopy and optical properties of reactive-sputtered silicon oxynitride films,” J. Phys. D: Appl. Phys., vol. 57, no. 28, 2024. https://doi.org/10.1088/1361-6463/ad3d6a.Search in Google Scholar

[4] H. Deviren and E. Çilgin, “Spectroscopic verification of biodiesel synthesis from turpentine tree oil through two-step chemical reactions: investigation of the use of synthesized biodiesel and acetone as fuel additives in diesel engines,” Energy Sources, Part A, vol. 46, no. 1, pp. 35–53, 2024. https://doi.org/10.1080/15567036.2023.2277363.Search in Google Scholar

[5] X. X. Han, R. S. Rodriguez, C. L. Haynes, Y. Ozaki, and B. Zhao, “Surface-enhanced Raman spectroscopy,” Nat. Rev. Methods Primers, vol. 1, no. 1, p. 87, 2022. https://doi.org/10.1038/s43586-021-00083-6.Search in Google Scholar

[6] Y. Huang, H. Xu, Y. Lu, and Y. Chen, “All-dielectric metasurface for achieving perfect reflection at visible wavelengths,” J. Phys. Chem. C, vol. 122, no. 5, pp. 2990–2996, 2018. https://doi.org/10.1021/acs.jpcc.7b10417.Search in Google Scholar

[7] Y. Li, et al.., “Inverse-designed linear coherent photonic networks for high-resolution spectral reconstruction,” ACS Photonics, vol. 10, no. 4, pp. 1012–1018, 2023. https://doi.org/10.1021/acsphotonics.2c01172.Search in Google Scholar

[8] P. R. Wiecha, A. Arbouet, C. Girard, and O. L. Muskens, “Deep learning in nano-photonics: inverse design and beyond,” Photon. Res., vol. 9, no. 5, p. B182, 2021. https://doi.org/10.1364/PRJ.415960.Search in Google Scholar

[9] P. R. Wiecha, A. Lecestre, N. Mallet, and G. Larrieu, “Pushing the limits of optical information storage using deep learning,” Nat. Nanotechnol., vol. 14, no. 3, p. 237-+, 2019. https://doi.org/10.1038/s41565-018-0346-1.Search in Google Scholar PubMed

[10] Q. A. Dong, et al.., “Plasmonic nanostructure characterized by deep-neural-network-assisted spectroscopy [Invited],” Chin. Opt. Lett., vol. 21, no. 1, pp. 1671–7694, 2023. https://doi.org/10.3788/col202321.010004.Search in Google Scholar

[11] S. Zhao, N. Liao, and B. Tan, “Overall design and data acquisition system of miniature color measuring spectrometer,” Opt. Technol., vol. 36, no. 6, pp. 848–851, 2010.Search in Google Scholar

[12] L. Zhang, M. Yang, Y. Zou, and C. Han, “Instrument line shape of infrared Fourier transform spectrometer and its engineer applications,” Opt. Precis. Eng., vol. 23, no. 12, pp. 3322–3328, 2015. https://doi.org/10.3788/ope.20152312.3322.Search in Google Scholar

[13] W. He, et al.., “Non-local meets global: an iterative paradigm for hyperspectral image restoration,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 44, no. 4, pp. 2089–2107, 2022. https://doi.org/10.1109/TPAMI.2020.3027563.Search in Google Scholar PubMed

[14] Y. Jahani, et al.., “Imaging-based spectrometer-less optofluidic biosensors based on dielectric metasurfaces for detecting extracellular vesicles,” Nat. Commun., vol. 12, no. 1, p. 3246, 2021. https://doi.org/10.1038/s41467-021-23257-y.Search in Google Scholar PubMed PubMed Central

[15] Z. Wang, et al.., “Single-shot on-chip spectral sensors based on photonic crystal slabs,” Nat. Commun., vol. 10, no. 1, p. 1020, 2019. https://doi.org/10.1038/s41467-019-08994-5.Search in Google Scholar PubMed PubMed Central

[16] L. Gao, Y. Qu, L. Wang, and Z. Yu, “Computational spectrometers enabled by nanophotonics and deep learning,” Nanophotonics, vol. 11, no. 11, pp. 2507–2529, 2022. https://doi.org/10.1515/nanoph-2021-0636.Search in Google Scholar

[17] Y. Zhu, X. Lei, K. X. Wang, and Z. Yu, “Compact CMOS spectral sensor for the visible spectrum,” Photonics Res., vol. 7, no. 9, pp. 2327–9125, 2019. https://doi.org/10.1364/prj.7.000961.Search in Google Scholar

[18] S. Wen, X. Xue, S. Wang, Y. Ni, L. Sun, and Y. Yang, “Metasurface array for single-shot spectroscopic ellipsometry,” Light: Sci. Appl., vol. 13, no. 1, p. 88, 2024. https://doi.org/10.1038/s41377-024-01396-3.Search in Google Scholar PubMed PubMed Central

[19] G. Cai, et al.., “Compact angle-resolved metasurface spectrometer,” Nat. Mater., vol. 23, no. 1, pp. 71–78, 2024. https://doi.org/10.1038/s41563-023-01710-1.Search in Google Scholar PubMed

[20] B. Cerjan and N. J. Halas, “Toward a nanophotonic nose: a compressive sensing-enhanced, optoelectronic mid-infrared spectrometer,” ACS Photonics, vol. 6, no. 1, pp. 79–86, 2018. https://doi.org/10.1021/acsphotonics.8b01503.Search in Google Scholar

[21] Y. Xiao, et al.., “Superconducting single-photon spectrometer with 3D-printed photonic-crystal filters,” ACS Photonics, vol. 9, no. 10, pp. 3450–3456, 2022. https://doi.org/10.1021/acsphotonics.2c01097.Search in Google Scholar

[22] J. Wen, et al.., “Deep learning-based miniaturized all-dielectric ultracompact film spectrometer,” ACS Photonics, vol. 10, no. 1, pp. 225–233, 2022. https://doi.org/10.1021/acsphotonics.2c01498.Search in Google Scholar

[23] C. Brown, et al.., “Neural network-based on-chip spectroscopy using a scalable plasmonic encoder,” ACS Nano, vol. 15, no. 4, pp. 6305–6315, 2021. https://doi.org/10.1021/acsnano.1c00079.Search in Google Scholar PubMed

[24] Conor C. Horgan, Magnus Jensen, Anika Nagelkerke, Jean-Philippe St-Pierre, Tom Vercauteren, Molly M. Stevens, Mads S. Bergholt, et al.., “High-Throughput Molecular Imaging via Deep-Learning-Enabled Raman Spectroscopy,” Anal. Chem., vol. 93, no. 48, pp. 15850–15860, 2021. https://doi.org/10.1021/acs.analchem.1c02178.Search in Google Scholar PubMed PubMed Central

[25] Zimeng Zhang, Yingjie Liu, Zi Wang, Yao Zhang, Xiaoyuan Guo, Shumin Xiao, Ke Xu, and Qinghai Song, “Folded Digital Meta-Lenses for on-Chip Spectrometer,” Nano Lett., vol. 23, no. 8, pp. 3459–3466, 2023. https://doi.org/10.1021/acs.nanolett.3c00515.Search in Google Scholar PubMed

[26] Z. Yang, et al.., “Single-nanowire spectrometers,” Science, vol. 365, no. 6457, p. 1017-+, 2019. https://doi.org/10.1126/science.aax8814.Search in Google Scholar PubMed

[27] J. J. Meng, J. J. Cadusch, and K. B. Crozier, “Plasmonic mid-infrared filter array-detector array chemical classifier based on machine learning,” ACS Photonics, vol. 8, no. 2, pp. 648–657, 2021. https://doi.org/10.1021/acsphotonics.0c01786.Search in Google Scholar

[28] W. Wang, Q. Dong, Z. Zhang, H. Cao, J. Xiang, and L. Gao, “Inverse design of photonic crystal filters with arbitrary correlation and size for accurate spectrum reconstruction,” Appl. Opt., vol. 62, no. 8, pp. 1907–1914, 2023. https://doi.org/10.1364/AO.482433.Search in Google Scholar PubMed

[29] M. Zhou, et al.., “Inverse design of metasurfaces based on coupled-mode theory and adjoint optimization,” ACS Photonics, vol. 8, no. 8, pp. 2265–2273, 2021. https://doi.org/10.1021/acsphotonics.1c00100.Search in Google Scholar

[30] E. Khoram, Z. Wu, Y. Qu, M. Zhou, and Z. Yu, “Graph neural networks for metasurface modeling,” ACS Photonics, vol. 10, no. 4, pp. 2330–4022, 2022. https://doi.org/10.1021/acsphotonics.2c01019.Search in Google Scholar

[31] M. Yako, et al.., “Video-rate hyperspectral camera based on a CMOS-compatible random array of Fabry–Pérot filters,” Nat. Photonics, vol. 17, no. 3, pp. 218–223, 2023. https://doi.org/10.1038/s41566-022-01141-5.Search in Google Scholar


Supplementary Material

This article contains supplementary material (https://doi.org/10.1515/nanoph-2024-0328).


Received: 2024-06-20
Accepted: 2024-08-14
Published Online: 2024-08-29

© 2024 the author(s), published by De Gruyter, Berlin/Boston

This work is licensed under the Creative Commons Attribution 4.0 International License.