Fig. 1. We present a multi-camera array system that rapidly captures large focal stacks of macroscopically curved objects at microscopic resolutions over a wide FOV. A novel reconstruction algorithm then fuses the acquired imagery into large-area all-in-focus composites and recovers the associated depth maps to provide RGBD scans with uniquely high resolution. We demonstrate our method by obtaining high-resolution RGBD images of curved objects up to 10 cm in size.

Fig. 2. Overview of the data acquisition process using the multi-camera array imager. (a) Hardware schematic of the multi-camera array, and more than 50% overlap in the FOVs of individual cameras allows imaging over a large continuous area. (b) The multi-camera array scans objects axially to create a focal stack for each camera with a total of 6×9=54 camera images per z slice. (c) Actual setup: The sample is mounted on a stage and illuminated using 4 LED panels placed symmetrically around the sample stage.

Fig. 3. Joint 3D reconstruction and stitching algorithm. There are two sets of losses to optimize the CNN. (a) Multi-view consistency losses. We use a CNN to convert captured focal stacks into camera-centric depth maps. These depth maps are used to create an AiF RGB image. Both the camera-centric depth map and RGB image are dewarped using calibrated camera parameters to create RGB and depth composites. The composites are rewarped to create forward predictions of camera-centric depth maps and AiF RGB images. The mean square error (MSE) between camera-centric and forward predictions is used to train the CNN. (b) Sharpness losses. The MSE between the depth map produced by the CNN and the depth from sharpest focus, and the negative sharpness of the composite created using the CNN-predicted depth map are also used to optimize the CNN.

Fig. 4. Characterization of the imaging system. (a) 3D-printed pyramid rendering. (b) All-in-focus composite image and predicted depth map. (c) Line profile of the sample along the green line in (b). (d) Packaging foam piece rendering. (e) All-in-focus composite image and predicted depth map. (f) Line profile of the sample along the green line in (e).

Fig. 5. High-resolution RGBD reconstructions of large objects. For each object, (a) shows the all-in-focus composite image, and (b) shows the depth map. Insets (c) and (e) are enlarged images of the all-in-focus composites, and insets (d) and (f) are enlarged images of the depth maps. Our method can identify small surface features like holes (A) and divots (C), effectively reconstruct objects with height variations much larger than the DOF (B), and is robust to variation in object material (D).

Fig. 6. 3D rendering using reconstructed RGBD data. (a)–(c) show 3D renders of an object imaged by our system in different lighting conditions. (d)–(f) show enlarged renders of the marked region under the same lighting conditions. Due to the high spatial and depth resolution afforded by our method, small surface variations can be seen easily.