7

Modeling my Grandpa with 3D Photogrammetry

 3 years ago
source link: https://www.jeffgeerling.com/blog/2021/modeling-my-grandpa-3d-photogrammetry
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.
neoserver,ios ssh client

Modeling my Grandpa with 3D Photogrammetry

August 25, 2021

Today I released a video about how—and why—I 3D Printed my Grandpa.

I sculpted a bust of my Grandpa in high school, gave it to my grandparents, got it back after he died and my Grandma moved out of her house (I wrote a tribute to my 'Grandpa Charlie'), and I kept on moving it around my office because I didn't have room for it:

Grandpa bust - terracotta by Jeff Geerling in 2001 - original statue

Grandpa by Jeff Geerling, terracotta, 2001.

I decided it had to go, but asked my extended family if anyone wanted the statue (thinking it would be sad to destroy it). One enterprising cousin suggested he could 'copy' the statue in smaller form using photogrammetry:

  • I'd take many pictures of the statue
  • He'd convert it into a 3D model using software algorithms
  • He'd print a smaller (and much lighter) copy of the bust for any takers

The video above goes into more detail about the entire journey, but in this blog post I wanted to give some detail about the photogrammetry process itself.

Jeff Geerling looks at Grandpa 3D Printed Ketchup Bottle Topper

Coming into it, it seemed like some form of magic. And after generating a highly-detailed 3D model from a physical object with nothing but my camera and my laptop, it still feels like magic. But at least it's highly deterministic magic that can be grasped.

Taking Pictures

Any Photogrammetry adventure begins with capturing photographs of a 3D object.

In my case, the object was a bust of my Grandpa, which weighed 16 lbs (7.26 kg) and was about 1 foot (30 cm) wide.

I could've set the bust on a surface and walked around taking pictures of it, relying on software to 'clean things up' for me.

But as a programmer, I know garbage in equals garbage out, and the cleaner the images, and the more precisely they are taken, the less work I'll have to do later cleaning up artifacts or touching up an imprecise model—at least that's my theory.

So I built a rotating turntable out of two round pieces of wood and a lazy susan bearing:

Assembling turntable or lazy susan using drill

I set it up with a pure white background (a sheet of posterboard), and measured out 10° increments on a piece of tape I put on the DIY lazy susan, so I could rotate it precisely.

Finally, I took a series of 148 pictures, from four elevations:

Grandpa bust spinning - photogrammetry

I pulled those pictures into the computer, tweaked the exposure a bit so the white background was blown out a tiny bit (pure white), and had a set of 24 megapixel JPEGs ready for the photogrammetry process—over 3.5 billion pixels of data to work with!

First Attempts (Failures)

Since I have a Mac (a 2019 16" Macbook Pro with an i9 processor at the time), I don't have any Nvidia CUDA cores at my disposal (locally, at least). And it seems some of the more GUI-driven photogrammetry programs (which often require CUDA cores) are Windows-only.

But I eventually found the free and cross-platform app Regard3D, and I followed the official tutorial. I was able to get a result, but the resulting 3D mesh was really messy:

Grandpa Regard3D blobby goo model

You could see my Grandpa, but he seemed to be frozen in a bed of goo. It would've required a lot of work in Meshmixer to get a faithful reproduction of the original bust.

I sent my cousin the same set of pictures, but was also having trouble getting a good initial model.

Second Attempt: COLMAP + OpenMVS

Finally, I found this blog post on COLMAP + OpenMVS from Dr. Peter L. Falkingham. He wrote a .bat script that uses the two open source tools (on Windows, at least) to generate a 3D model from a set of images.

I needed to install COLMAP and OpenMVS, and luckily, I also found an excellent Instructable by joecooning, Free Photogrammetry on Mac OS: From Photos to 3D Models, and followed his instructions to install them:

Install COLMAP

Download the latest COLMAP-dev-mac-no-cuda.zip file from the COLMAP GitHub releases page, expand it, and place COLMAP in the Applications folder.

Install OpenMVS

  1. Install OpenMVS' dependencies with Homebrew: brew install boost eigen opencv cgal ceres-solver.
  2. Install CMake.
  3. Install XCode.
  4. Clone the VCG Library: git clone https://github.com/cdcseacave/VCG.git vcglib
  5. Clone OpenMVS: git clone https://github.com/cdcseacave/openMVS.git
  6. Create a separate build directory to build OpenMVS: mkdir openMVS_build && cd openMVS_build
  7. Build OpenMVS: cmake . ../openMVS -DCMAKE_BUILD_TYPE=Release -DVCG_ROOT="pwd/vcglib" -G "Xcode"
  8. Use xcodebuild to compile the app: xcodebuild -configuration Release

Run photogrammetry.sh

Use this shell script (embedded below) to run the photogrammetry process inside a directory full of images of the object you wish to turn into a 3D model:

#!/bin/bash # Photogrammety automation script. # # Based on https://peterfalkingham.com/2018/04/01/colmap-openmvs-scripts-updated/ # Adapted from https://www.instructables.com/Free-Photogrammetry-on-Mac-OS-From-Photos-to-3D-Mo/ # # See full guide on Jeff Geerling's blog: # TODO # # Usage: # 1. Save this script into a file named `photogrammetry.sh`, in the directory # where you saved the photos you took of a 3D object. # 2. Give it execute permissions (`chmod +x photogrammetry.sh`). # 3. Replace the '/Users/jgeerling/*' paths with your own. # 4. Run `./photogrammetry.sh` # 5. Wait. # # It will take a LONG time, especially if you use tons of high-res photos.

# Get current directory path. current_dir=$PWD

# Store the current directory name for object output. dir_name=${PWD##*/}

# Set colmap directory (change this to where you've placed colmap). colmap_dir=/Applications/COLMAP.app/Contents/MacOS/colmap

# Set openMVS directory (change this to the 'bin/Release' folder where you built openMVS). openmvs_dir=/Users/jgeerling/Downloads/photogrammetry/openMVS_build/bin/Release

# Set Working Directory (I create a temporary workspace folder in my 'Projects' directory to process data in) working_dir=/Users/jgeerling/Downloads/photogrammetry/workspace

mkdir $working_dir cp *.jpg $working_dir/ cp *.JPG $working_dir/ cd $working_dir

$colmap_dir feature_extractor --database_path database.db --image_path . $colmap_dir exhaustive_matcher --database_path database.db mkdir sparse $colmap_dir mapper --database_path database.db --image_path . --output_path sparse $colmap_dir model_converter --input_path sparse/0 --output_path model.nvm --output_type NVM $openmvs_dir/InterfaceVisualSFM model.nvm $openmvs_dir/DensifyPointCloud model.mvs $openmvs_dir/ReconstructMesh model_dense.mvs $openmvs_dir/RefineMesh --resolution-level 1 model_dense_mesh.mvs $openmvs_dir/TextureMesh --export-type obj -o $dir_name.obj model_dense_mesh_refine.mvs

mkdir $current_dir/model/ cp *.obj $current_dir/model/ cp *.mtl $current_dir/model/ cp *Kd.jpg $current_dir/model/

cd $current_dir

After running the script, the new 'model' directory should contain a .jpg texture map image, a .mtl (material settings) file, and an .obj (3D object) file.

I imported the .obj file in the free Meshmixer app, and WOW, it was pretty much dead-on... besides the fact that the object was upside-down:

COLMAP OpenMVS grandpa object imported into MeshMixer

The Final Result

So I spent a bit of time cleaning up the bottom edges in Meshmixer, and doing a 'plane cut' to slice off the bottom and discard all the artifacts from the lazy susan/turntable. Then I made the object a solid (this isn't strictly necessary for 3D printing, but it's easier to work with in my case) and exported it.

It looks pretty amazing, even capturing some of the tiny details that the naked eye would likely miss at a glance!

Final Grandpa statue in Meshmixer from COLMAP and OpenMVS

Here's one brief clip of a 3D print I made of this model (generated with Octolapse and my Nikon D700):

Octolapse timelapse of 3D Printing my Grandpa in white PLA with clock in background

To see a ton more detail about the story behind this bust, and the different designs I made for my family, please watch the video.


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK