Underwater Imaging for Coral Reef 3D Mapping

Background

The Viseaon Marine Imaging Lab’s research focus is on developing imaging systems and computer-vision algorithms to explore the ocean and its inhabitants. Research in the lab varies from underwater microscopy to wide-scale 3D mapping, and from vision-based navigation and fabrication of imaging systems for autonomous submarines to deep-learning for ecological research and monitoring (shark counts, coral identification). We are based in the Israeli Oceanographic and Limnological Research (IOLR) institution in Haifa, Israel. The lab is a part of the Hatter Dept. of Marine Technologies, in the Charney School of Marine Science, University of Haifa, Israel.

We started working on underwater 3D modelling using photogrammetry in 2017. At the time, diver-based photogrammetry was emerging as an attractive method for marine surveys. It was being studied and implemented by several research groups around the world, and we were inspired by other researchers such as John Burns and Renata Ferrari and projects such as the Catlin Seaview survey and the 100 Island Challenge.

Our main goal was to create large photorealistic 3D maps of the seabed and use them to study coral reef organisms. We began to experiment with different methods for image acquisition and processing, slowly developing our own style and workflow [1].

Coral reefs

Coral reefs are some of the most important ecosystems on the planet. Not only do they provide essential habitat for other reef organisms, but also shoreline protection, resources, and employment. Corals are colonial organisms of the phylum Cnidaria. They grow in elaborate forms, building the coral reef ecosystem. Although many studies have focused on the biology of these organisms, much remains to be learned on their community ecology (studying groups of populations). Systematic research of coral reefs is very difficult because they are some of the richest and most intricate environments on the planet. Corals often grow on top of each other, and change their shape in response to environmental conditions (phenotypic plasticity). Therefore, measuring an individual coral colony is very difficult. Traditionally, reef surveys have been conducted by divers with measuring tapes. Photogrammetry and 3D modeling is useful for studying coral reefs because the models produced are detailed, accurate, and can be applied across large areas. Moreover, in coral reefs, 3D structural-complexity relates to biological diversity, and 3D imaging enables us to examine this connection from the computer, rather than in the water while diving where we are limited by breathing gas and depth. We also found that 3D imaging is very useful for modeling shipwrecks. After experimenting on shipwrecks in the Eastern Caribbean and northern Red Sea, we received support from the Israeli Dive Federation to map several shipwrecks in the eastern Mediterranean under the project Shipwreck Israel.

Workflow

Our workflow consists of scuba diving to a selected site followed by acquiring images of a selected reef or area. We then return to the lab and use Agisoft Metashape software  to process a 3D model from the acquired images. The models are then analyzed for their 3D shape and coral colony composition. We use a DSLR camera, strobes, scale-bars, a leveler, and SCUBA diving equipment. For further reading please see [1]. The best tip we can give for a successful 3D reconstruction is to make sure that the images are illuminated and sharp.

Research goals

Our research goals are on several fronts. One aspect of our work concerns automatic coral identification and segmentation of 3D models. Another aspect concerns exploring the 3D shape of coral reefs to understand the overarching principles that govern coral reef geometry. Overall, we would like to develop fully automated pipelines for underwater semantic mapping (assigning each cell in a map with a meaningful biological classification) that would become useful in research and conservation.

Challenges

Some of the main challenges in our work are in underwater imaging and automated 3D data analysis. To solve these problems, our team is developing visibility enhancement solutions as well as automated underwater 3D segmentation and geometrical analysis. For example, the Sea-Thru algorithm developed by Akkaynak and Treibitz [2] provides true-color correction of underwater images, resulting in much better 3D reconstructions that enable us to identify corals more easily.

Working with Sketchfab

We found Sketchfab as a great platform for viewing and sharing the 3D models online very early on. Sketchfab enables viewing models in Virtual Reality (VR) which is a great feature that we use to show underwater 3D models to visitors in the lab. The annotation tool is very useful, as it lets us determine selected points of view, and navigate easily through a large model.

Future prospects

We started working with 3D because we saw the potential of 3D imaging for coral reef ecological research and conservation. 3D technologies have far reaching implications for the scientific community. They are effective for communicating scientific results in a captivating manner, and offer new ways to explore and analyze data. With developments in hardware and software, we expect that VR will be used extensively for different tasks. For example, it can be used for communication such as in social media and virtual conferences, for operating robots remotely, and for scientific data analysis such as 3D segmentation.

Due to increasing levels of global and local stress, coral reefs are changing their shapes and are becoming flatter and less complex, causing, for example, a decrease in shelter for fish and other reef organisms. An important aspect of 3D imaging is that it enables us to depict this process which is otherwise very difficult to quantify. We hope that this data together with powerful visualization tools (VR) can influence sustainable environmental policies.

References:

[1] Yuval, Matan, et al. “Repeatable semantic reef-mapping through photogrammetry and label-augmentation.” Remote Sensing 13.4 (2021): 659.

[2] Akkaynak, Derya, and Tali Treibitz. “Sea-thru: A method for removing water from underwater images.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.

Acknowledgements:

We thank D. Akkaynak, A. Avni, and D. Levy, E. Rozen and the Morris-Kahn Marine Research Station, the Hatter Dept. of Marine Technologies, the Dept. of Marine Biology, Univ. of Haifa, the Inter-University Institute for Marine Sciences of Eilat and all divers, the Israeli Dive Federation and S. Palnitzky, the Caribbean Netherland Science Institute and ASSEMBLE+, the Acadia S.V., the Rohr Reef Resilience team, D. Kline and the Smithsonian Tropical Research Institute, Microsoft AI for Earth,  the Murray Foundation, the PADI Foundation, Agisoft Support team, the Israel Ministry of National Infrastructures, Energy and Water Resources, the Israel Ministry of Science, Technology and Space, and the University of Haifa Data Science Research Centre. We thank G. Banc-Prandi for image 2. This communication is dedicated in memory and honour of Andy Phillips, an inspirational diver and PADI course director.