Digitizing Biodiversity: Capturing the Aquatic Creatures of Asia in 3D

About me

It’s an honor to introduce myself on this blog. My name is Yuichi Kano, I’m Japanese and living in Fukuoka City (the best place for “Ramen” noodle soup), in the southern part of Japan. I am an ecologist at Kyushu University, mainly studying the inland-water ecosystem of Monsoon Asia, one of the biodiversity hotspots of the world. If you’re interested, please see my best scientific paper about the impact of hydropower dams on fish in the Mekong Basin.

By the way, I have deeply loved nature and wild fauna/flora since I was a kid, and I have been interested in photography and recording movies/sounds of them in the fields.

Since I became a researcher, as a subset of my work, I have started to archive multimedia digital information (photos, sounds, movies, DNA sequences, etc.) about biodiversity that I have met in the fields, using my programming skills (see the ffish.asia website). Recently, I’m addicted to 3D modeling biological specimens using photogrammetry. It’s lots of fun for me, honestly—more exciting than general scientific studies.

Why photogrammetry?

My first 3D model made with photogrammetry is the water bug Diplonychus rusticus.

This bug had been thought to be already extinct in Japan, but I found it in Japan and this is its first rediscovery in 56 years. I strongly believed that the treasured specimen should be published online as a 3D model. Until then, I had archived 3D models of biological specimens by CT scanner (e.g., in this article). However, the CT scanner was broken. So I tried 3D photogrammetry using my digital camera. The result was much more excellent than I had expected. It was actually better than the CT scanning (which requires very expensive machines and softwares!). I also noticed that the textured 3D models made with photogrammetry (CT scanned models have no color) were attractive to the public and had a high affinity with open science.

Equipment and camera settings

For photogrammetry, my method is very simple. I use a Nikon D7500 and a black panel as the background. To ensure even lighting, a remote flash set is also used (Nikon SB-700 and SU-800). The object specimen is hung by nylon string(s) and photos are taken from many angles.

In photography, the depth of field (F-value) is set to the maximum (25-40; depending on the lens) with the strongest flash (+3.0). The ISO is set at 500-1200 according to the specimen color: dark-colored specimen needs a higher ISO. I use 3DF Zephyr for the 3D model generation. I have never used other software and do not know whether the other software is better or worse, although I feel 3DF Zephyr is a pretty nice (and low cost!) software. Sometimes, the black background may not be needed as the software conveniently ignores the unfocused areas.

Reconstruction of biological specimens

My major interest is fish. However, it is very difficult to precisely generate thin parts such as fins and barbels. In my first attempts, I could not generate good models. For example:

This is because the fins dried up and curled while I was taking photos; they are also semi-transparent, which makes it difficult for the software to reconstruct them. I discovered, however, that fins do not dry up if the specimens are cooled in ice water for 30-60 min (after 30-120 formalin fixation). I also found that ice water made the specimen more rigid and a little bit more colorful.

For these models, it has been important to take more than 500 photos and then select the best 500 photos (the maximum number that can be read by the software). Even with that much photographic coverage, the reconstructed meshes of the fins sometimes have many small holes. The filter “Fill Holes – WaterTight” is effective for such holes: the filter can be applied again and again until all the holes are filled (3-5 times). As a result, more precise fish models can be generated, like:

As for invertebrates with many stringlike parts, it is important to take photos that focus on the front edges of the strings, as well as photographing the maximum number of photos and angles.

I have only one year of experience with photogrammetry—I am really just a beginner. So I do not know more advanced techniques of photogrammetry and ancillary softwares (e.g., Blender). I still don’t have the skill to generate models of transparent organisms such as shrimp: it is my next challenge.

Textured 3D models for open science, AI, and museology

As I stated above, textured 3D models have quite a high affinity with public open science and environmental education. People can use these 3D models for species identification and they can be more useful than 2D photo images for this purpose. More than anything, people can become interested in biodiversity (and its conservation) through realistic 3D models. Sketchfab, which is easily accessible to the general public, is very useful as a repository for these models. I can even embed my models on my website (e.g. the Mauremys japonica)!

Further, I recently embarked upon a challenge to apply the textured 3D models to AI species identification. Textured 3D models contain a lot of information compared to 2D photographs, so AI can “learn” a huge amount of information from just one 3D model.

Textured 3D models would be also useful in museology. We can digitally back up the important specimens, such as type specimens, using photogrammetry.

This approach enables taxonomists to easily access a specimen even if they are unable to go to the museum. Also, the easy online access to specimens increases people’s opportunities to learn about the existence of scientific specimens. At least in Japan, museology has suffered from a lack of funds, resources, and interest. For the general public, sometimes even for scientists, museums and museology are rather unfamiliar. Publishing important and precious or beautiful specimens (or any collection) from museums may be one of the next new steps in the development of Japanese museology.