Fig. 1. FiLM-Scope configuration and example images. (a) Top: picture of multicamera array microscope (MCAM) lenses. 48× lenses are held in a single block, with 9 mm pitch. Bottom: 3D model of FiLM-Scope. (b) Snapshots from the FiLM-Scope of an ex vivo rat skull. 48 images are acquired in a single frame, allowing for video acquisition at up to 120 frames per second. The green and purple insets highlight the angular range of the system, whereas the red inset highlights the high resolution. (c) 3D height map from the FiLM-Scope. Top left: single image of the rat skull from the FiLM-Scope. Top-right: height map for the rat skull. Bottom: skull displayed as a 3D point cloud, generated using the height map. (d) Four frames from a video of a hand-held tool moving over the surface of a rat skull, acquired at 100 frames per second. 3D reconstruction was performed on all four frames, and the resulting point clouds are overlayed in the bottom frame, highlighting the motion of the tool in 3D space.

Fig. 2. FiLM-Scope calibration results. (a) Diagram of ideal parametric FiLM-Scope system, demonstrating how 3D information is encoded in the images. Red points lie on object plane and appear at the same pixel location in both highlighted elemental images. Blue points are axially translated off the object plane. On the center elemental image, the blue points project to the same image location as the red points (forming purple points). In the upper elemental image, the projections of the blue points are laterally translated relative to the projections of the red points. (b) Example FiLM-Scope calibration results, showing shift slopes (Si) for all 48 cameras (all units: pixels/mm). Top: average shift slope value for each camera. Bottom (red and blue insets): Varying Si(p) values for two cameras across the image fields-of-view. (c) Top: on-axis, diffraction-limited axial resolution for each camera with a logarithmic scale bar. Bottom: on-axis, diffraction limited lateral resolution for each camera. (d) Digital refocusing results with raw reference camera image (top) and digital refocus back-projections of all 48 images to the respective axial planes (0, 6.4, 8.0, and 10.7 mm) using data from a snapshot in Fig. 1(b). A full digitally refocused volume is shown in Video 1 (Video 1, Mp4, 19.4 MB [URL: https://doi.org/10.1117/1.APN.4.4.046008.s1]).

Fig. 3. Diagram for self-supervised 3D reconstruction algorithm developed for the FiLM-Scope. Images are first back-projected into a 3D volume, using our custom calibration approach. The volume is then fed through a 3D U-Net, outputting a probability volume from which we can extract a height map. All images are rectified to a reference viewpoint using this height map, and the loss between each of the rectified images and the “reference image” (acquired from the reference camera) is used to update weights on the U-Net.

Fig. 4. Select results. (a) Images and height maps for an ex vivo rat skull. For each of the four crops, the left panel shows the grayscale image, and the right panel shows an overlay of the reconstructed height map and the grayscale image. The FiLM-Scope algorithm detects small height changes on the skull surface, down to fractions of a millimeter. (b)–(c) Grayscale images and height maps of human fingers. Insets highlight details of the reconstructed height map. In the bottom inset for each image, the global tilt in the height map is removed to clearly highlight 3D spatial resolution.

Fig. 5. FiLM-Scope video acquisition. (a) 3D tracking of points on a tool, as a human user moves the tool over the surface of an ex vivo rat skull. Top: select frames with marked points. Bottom: graph of estimated heights for each point during 8 s of acquired video, captured at 100 FPS, along with the angle of the tool, measured between two points on its surface. The full video is provided in Video 2 (Video 2, Mp4, 16.4 MB [URL: https://doi.org/10.1117/1.APN.4.4.046008.s2]). (b) Estimated height of tool translated axially on an automated stage away from an ex vivo rat skull. Our algorithm accurately estimated the height of the tool with 11μm standard error. (c) Select frames from a video and corresponding height maps of human skin. Video was acquired with 4×4pixel binning at 100 FPS. In the right-most insets, global tilt was removed from the height maps to highlight fine features. The full video is shown in Video 3 (Video 3, Mp4, 38.2 MB [URL: https://doi.org/10.1117/1.APN.4.4.046008.s3]).