Trace detection is common in criminal investigation[
Chinese Optics Letters, Volume. 19, Issue 3, 031101(2021)
Reflectance transformation imaging of 3D detection for subtle traces
This paper reports a detection method of two-dimensional (2D) enhancement and three-dimensional (3D) reconstruction for subtle traces with reflectance transformation imaging, which can effectively locate the trace area of interest and extract the normal data of this area directly. In millimeter- and micron-scale traces, during 3D construction, we presented a method of data screening, conversion, and amplification, which can successfully suppress noise, improve surface and edge quality, and enhance 3D effect and contrast. The method not only captures 2D and 3D morphologies of traces clearly but also obtains the sizes of these traces.
1. Introduction
Trace detection is common in criminal investigation[
In past decades, 3D imaging has been the interest of researchers. Frankot and Chellappa[
However, most of these common 3D methods focus on the reconstruction of macroscopic objects. There are few reports on the 3D reconstruction of faint traces of millimeters and microns. Therefore, in this paper, we use RTI to enhance the weak traces on the front and back of metal and obtain the surface normal data at the same time. Based on these normal data, the 3D morphologies on both sides of the metal are reconstructed, and the depth information is captured. This method can obtain not only the 2D image of weak traces clearly but also the 3D morphology and 3D information, which provides a powerful tool for archaeology and criminal investigation.
Sign up for Chinese Optics Letters TOC Get the latest issue of Advanced Photonics delivered right to you!Sign up now
2. Method
The reflection characteristics of the object directly affect its realistic rendering, so the brightness of each pixel of the object image is related to the direction of the incident light. RTI models this dependence by a biquadratic polynomial:
Figure 1 shows the image acquisition device, which is composed of a fixed digital single lens reflex (DSLR) camera (Canon EOS 5D Mark IV) with a resolution of pixels directly above the detection plane and a motor-driven RTI half-arc with a radius of 0.5 m. Nine LEDs (Cree XLamp XM-L), lit in turn, facing the right center of the detection plane, are fixed on the RTI arc at equal angle intervals (15°). The RTI arc rotates six times by 30° in turn from the angle 15° with respect to the detection plane. Simultaneously, the camera (1/25 s, F 8.0, ISO 250) captures 54 images with a small black ball to record the direction of the incident lights (red box in Fig. 1), which can then be calculated from the relative positions of the highlights. Then, we can solve the coefficients of the biquadratic polynomial for each pixel by singular value decomposition (SVD):
Figure 1.Image acquisition device and the distribution of the direction of the incident light.
With the polynomial coefficients of each pixel, we can reconstruct the image under any virtual illumination direction, providing convenience in capturing the panorama of traces. After deriving Eq. (1), we can get the normalized surface normal :
These noramls are stored in a normal map, where the RGB values of each texel represent the normal components X, Y and Z. X and Y are in the range , and Z is in the range [0,1]. To facilitate storage, these data are mapped to [0,255], forming a normal map.
Depth recovery by gradient integration first assumes that the surface depth of the object is , and then the gradients , and the partial derivative of the gradients , are defined as follows[
Then, we can obtain the error equation:
Therefore, the problem of solving the optimal depth information is shifted into finding the minimum solution of the continuous function:
Conventional algorithms focus on restoring the 3D shape of a macroscopic object. However, the traces detected are only a few hundreds or dozens of microns in size, and the noise introduced in the measurement has a great influence on the recovery of 3D morphology. Therefore, we proposed a data processing method, which implements the surface normal data into the surface gradient data, magnifies the micro change of the gradient, and reduces the height of the accumulated error.
Firstly, we convert the normal of each pixel as follows:
In this step, the normals are converted into gradient data of the surface height to facilitate depth recovery; moreover, slight surface changes can be amplified by the weight coefficient .
Secondly, to avoid some gross errors, we select threshold to screen and . If or , we set them to zero.
Then, we choose a local window of size with each pixel of the gradient map as the center to calculate the mean value of the window as a reference value and amplify the gap between the traces and the background to enhance the visual effect of 3D reconstruction:
After that, we obtained the new gradient maps P and Q of the surface with the size of , which we extended to avoid the periodicity problem of the Fourier transform:
Finally, the depth information of trace is recovered by a global integral[
3. Results and Discussions
A metal sheet with Chinese characters printed on its surface was selected as the experimental sample. Because of oxidation and corrosion, it is difficult to get the information of these characters directly and clearly; there is almost no trace on its back [Figs. 2(a) and 2(c)]. The normal maps of these two surfaces are shown in Fig. 3. Since the metal sheet is a plane, and its normal almost points to the direction of the Z axis, the normal maps are nearly all blue, and part of the color change represents the change of the normal, which indicates the trace information. These maps also show that the changes in normal of traces are weak. RTI with an enhanced algorithm helps capture the trace information easily and accurately [Figs. 2(b) and 2(d)]. The selected red box in Fig. 2(b) shows the most complicated structure of the Chinese character clearly. Then, after this part of the normal data is extracted, the depth recovery algorithm is tested.
Figure 2.Sample and RTI reconstructions. (a) Original photo of the front. (b) RTI reconstruction of the front with specular enhancement. (c) Original photo of the back. (d) RTI reconstruction of the back with specular enhancement.
Figure 3.Normal maps of the two sides of the metal in Fig.
Figure 4 is the result of the 3D reconstruction of the trace in the red box in Fig. 2. Figure 4(a) is the result from the common photometric stereo (PS) algorithm without filtering normal data, where there are some isolated peaks, indicating that there are some gross errors in the gradient data that need to be eliminated. Figure 4(b) shows the recovery using the PS algorithm () after setting the threshold to screen the normal data. The burr noise is eliminated, but the whole surface is not flat enough. Then, as shown in Fig. 4(c), we set to convert the normal data and set again for 3D recovery, which improves the overall face shape and edges. Finally, as shown in Fig. 4(d), we get a better shape, clearer edge, and lower height error with our method.
Figure 4.3D recovery results of the trace in the red box in Fig.
Meanwhile, as shown in Fig. 4(e), we intercepted the same flat parts of the recovery results, marked them with different colors, and calculated their mean square error (MSE) to indicate flatness. It is not difficult to find that the flatness of the surface recovered by our method is better, and the accumulated height error is also lower. Moreover, we also captured the relative depth of the same edges of the traces recovered by the above methods along the directions indicated by the colored arrows, as shown in Fig. 4(f). The result confirms the noise in Fig. 4(a) and illustrates that our method can effectively reconstruct the contour of the trace and reduce the accumulated height error. In short, proper and can improve the surface and edge of reconstruction.
In experiments, we found that and were good enough to rebuild the 3D shape, so we kept both settings for the rest of the experiment. Meanwhile, by adjusting the magnification factor and window size , we can get different 3D effects and contrast. For Fig. 2(d), as different and were applied, the effects are shown in Fig. 5. It is not difficult to find that proper enlargement of and can effectively help us observe the 3D structure of faint traces. Figure 5(a) shows the 3D shape without enhancement, where the trace and background are partly blended. As we increased and , the trace and background were efficiently separated, and the edge of the trace was significantly magnified [Figs. 5(b)–5(d)]. The 3D reconstruction of the metal back is shown in Fig. 6(a); in contrast with Fig. 2(c), some traces on the surface are faintly revealed, whereas, since the metal is much larger than traces, these traces are somewhat submerged. Instead, we only focused on the traces on the surface and enhanced the 3D reconstruction after capturing the regions of the traces from the 2D enhancement results. As shown in Fig. 6(b), the 3D information of these traces is effectively extracted.
Figure 5.Different enhanced 3D topographies of the trace marked by red arrows in Fig.
Figure 6.(a) Metal back 3D reconstruction (w = 0.5, k = 25, d = 25, λ = 1). (b) Trace 3D reconstruction on the back (w = 0.5, k = 25, d = 25, λ = 1).
Moreover, along the direction of the red arrows in Figs. 2(b) and 2(d), we scanned both parts by a DetakXT profilometer, and the results are shown as black curves in Fig. 7. Given the fixed relationship between pixel size and physical size, we set for 3D reconstruction of these parts, obtaining the altitude lines of their longitudinal sections. Due to the high precision of the DetakXT profilometer, we can evaluate the depth recovery by calculating the degree of coincidence between the depth recovery curve and the scanned curve. Therefore, we refer to the value:
Figure 7.Scanning results with the DetakXT profilometer (128 µm scanning range, 1% accuracy) and depth recovery results of the same part. The scanning position and direction are shown as red arrows in Fig.
As shown in Fig. 7, the recovery results of the parts with gentle changes were basically identical with the scanning results, while there were differences in the parts with abrupt changes, which is exactly what we predicted. Because of the properties of biquadratic polynomials, the high-frequency parts will be partially smoothed. Some other reasons, such as the approximate parallelism of beams and the limited number of sampling points, may lead to errors. Fortunately, only in the sharp parts is the error slightly larger, but also within 10 µm, and the depth information of interest is still available.
4. Conclusion
First of all, we describe the basic principle of RTI, and, then, for the faint traces below the millimeter level, we propose an optimized 3D reconstruction algorithm, which can effectively suppress the noise, improve the edge and surface shape, and enhance the 3D morphology. Next, the oxidized metal sheet was tested using the RTI device, and the traces on the metal surface were clearly visible in the 2D image with the specular enhancement. Finally, 3D reconstruction and depth information calculation are carried out on the weak traces. The results show that our method can realize the 3D reconstruction of tens of micron depth traces, and the error of depth recovery is less than 10 µm. Promisingly, our method has an important application prospect in the fields of cultural relic protection and criminal investigation.
[2] S. T. Porter, N. Huber, C. Hoyer, H. Floss. Portable and low-cost solutions to the imaging of Paleolithic art objects: a comparison of photogrammetry and reflectance transformation imaging. J. Archaeol. Sci.: Rep., 10, 859(2016).
[3] M. Tom, G. Dan, W. Hans. Polynomial texture maps. Conference on Computer Graphics & Interactive Techniques, 519(2001).
[5] M. Elfarargy, A. Rizq, M. Rashwan. 3D surface reconstruction using polynomial texture mapping. International Symposium on Advances in Visual Computing, Advances in Visual Computing, 353(2013).
[6] R. T. Frankot, R. Chellappa. A method for enforcing integrability in shape from shading algorithms. IEEE Trans. Pattern Anal. Mach. Intell., 10, 439(1988).
[7] D. Wang, Y. Meng, D. Chen, Y. Yam, S.-C. Chen. High-speed 3D imaging based on structured illumination and electrically tunable lens. Chin. Opt. Lett., 15, 090004(2017).
[12] W. Song, Q. Cheng, Y. Liu, Y. Zheng, Z. Lin, Y. Wang. Three-dimensional image authentication using binarized. Chin. Opt. Lett., 17, 051002(2019).
[15] L. Xu, M. Rahmani. Surface that perceives depth: 3D imaging with metasurfaces. Adv. Photon., 1, 030501(2019).
[16] W. Tiangong, K. Reinhard. A new algorithm for gradient field integration. IVCNZ 2001, 6(2001).
Get Citation
Copy Citation Text
Jie Dai, Lihua Huang, Kai Guo, Liqing Ling, Huijie Huang, "Reflectance transformation imaging of 3D detection for subtle traces," Chin. Opt. Lett. 19, 031101 (2021)
Category: Imaging Systems and Image Processing
Received: Nov. 9, 2020
Accepted: Jan. 12, 2021
Published Online: Feb. 25, 2021
The Author Email: Lihua Huang (hlh@siom.ac.cn)