3D Scanner – Software Overhaul

So I’ve been having a tinker with the 3D scanner again and I’ve got it working with my Cannon SX200 + a custom CHDK remote trigger script. This means the actual scan time is now about 3/4 of an hour and I only get about 2 scans per camera charge, but! (and this is a big but) I get glorious 12 mega-pixel resolution and awesome image quality.

This has meant I’ve had to have a rather massive overhaul of the model generating software to reduce memory usage, add more controls to the GUI, remove the OpenCV dependency, and generally make it more usable. It’s not finished yet, but here’s is a look at an early version of the new GUI:

Still need to beautify the layout a bit and write a considerable chunk of code. But it hopefully should be worth it.

As always the code is open source and available from the repository, but be warned the current committed version is massively broken, so if you want a working version checkout revision 2 (yeah, I know it’s probably terrible practice to break the main repo’s code, but what the hell, it’s not like anyone besides me is actually using this stuff).

Timeslice Code Available

Set up a google code svn repo for the time slicing code for anyone interested:
https://code.google.com/p/videotimeslice/

3D Scanner – C++ Code Available

The code is now available via svn. The URL is https://code.google.com/p/splinesweep/. Be warned! the code is awful….i mean really really pre pre alpha awful. Compile at your own risk, use at your own risk, and feel free to contribute if you can make any sort of sense out of the code (give me a shout first). The code is developed under netbeans, and is therefore structured as a netbeans project, there is probably a much more elegant way of setting up the svn so it doesn’t matter what IDE you use, but I don’t know it….so unfortunately it’s not set up in any such way.

In order to generate the models you’ll need 2 sequences of 100 images taken at 3.6 degree intervals around the object in question (atm its fixed at 100, sucks i know,will be changing this very shortly), one for the texture and one for the mesh points. They’ll probably look very much like this:

The Input Images

Start up the splinesweep program and click “Load Images” and select all the 100 laser spline images, then click “Load Textures” and select all 1oo full colour images. Hit “Generate Model” wait for a bit, and a .obj , .mtl and .png images should be created in the folder that the splinesweep program is located in.

SplineScan Screenshot

There is one pretty major limitation at the moment that I’ll probably get round to fixing within a week, currently the program assumes the centre of rotation of the platform to be at point that’s sort of at the middle of the image, and unfortunately this can only be changed within the code.

A few notes on dependencies:

The program requires QT4 and OpenCV2.0, but OpevCV1.0 may work, all-though the program does expect to find the opencv libraries in /usr/local/lib as opposed to /usr/lib. Soon as Ubuntu 10.04  is released i’ll build a version that works nicely with things you can apt-get.

How the code works:

First a sobel filter is run on all the spline images to produce edge images, then a threshold is taken of all the images to produce a binary image. To produce the 3D points for the mesh a series of points up the height of the image are taken and moved along the length of the image until a maximum pixel value is found or the edge of the image is reached, if the edge of the image is reached the point is discarded. The 2D points for each image are then placed in a rotated fashion around a central 3D axis to produce the 3D point cloud.

Holes in the point cloud are then filled in and a surface mesh is calculated. The RGB values in the colour images of every corresponding 2D point are found then placed into an image to create the texture, and the mappings between the 3D points and points in the texture are calculated.

This is all then written out to 3 files, an obj file which stores the vertexs,faces, and texture UV coordinates, a .mtl file which lists and describes properties of the materials used to texture the model, and  a .png file which is the actual texture.

There’s probably a bunch of little things going on that I can’t remember but this is as near as damnit to what’s actually going on.

3D Scanner – Textured Models!

Here is the nodding monkey with his texture on.

Textured Monkey

As you can probably see the texture is not that detailed. Each of the small faces that make up the model is textured with only 4 pixels worth of information. I’m going to try and work out a way to get a more detailed texture using the images captured during the scan (there’s a hella lot of information being thrown away creating the texture…admittedly a lot of it is repeated info…but there’s definitely enough to get a way better texture).

Edit: Here’s an animated gif of the scanning process. The laser-line and lights should alternate every frame, but i couldn’t be bothered ordering the frames in the gif.

Edit 2: Mesh smoothed with Laplacian filter and a white balance performed on the texture. Some of the finer details of the scan are lost due to the filter, but it does provide nice smooth surfaces. The texture seems slightly twisted to one side for some reason.

Time Slicing part 2

So here is a new time-sliced video and the opencv code used to create it. The video is under a Creative Commons Attribution-Non-Commercial-Share Alike 2.0 UK license and the code is GPL’ed.

In order to get the code to work you will need the opencv image processing libraries as well as the libboost libraries. The code was built and tested on Ubuntu 9.10 using Opencv 2.0 (not available in repos, will have to build yourself. Earlier versions of opencv probably should work.)

In order to use the program you will have to split out the individual frames of the video using this command:

ffmpeg -i filename.ogv -r 30 -f image2 %03d.jpg

Then run TimeSlice in the directory thusly:

./timeslice *.jpg

This will produce the individual time sliced frames (Overwriting some of the original frames). Now delete the remainder of the original frames (if there are any) and run the following command to create a .avi:

ffmpeg -sameq -r 30 -b 7200 -i %03d.jpg test.avi

/*
* File:   main.cpp
* Author: matt
*
* Created on 03 March 2010, 15:51
*/

/*This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <http://www.gnu.org/licenses/>.*/


#include <stdlib.h>
#include <cv.h>
#include <cvaux.h>
#include <highgui.h>
#include <iostream>
#include <iomanip>
#include <boost/intrusive/list.hpp>
using namespace std;

/*
*
*/
int main(int argc, char** argv) {

std::vector<IplImage*> image_vector;

//Load images
cerr << "Loading Images" << endl;
for (int loop = 1; loop < argc; loop++) {
IplImage *image;
cerr << "    Loading: " << argv[loop] << endl;
image = cvLoadImage(argv[loop]);

image_vector.push_back(image);

}
//return 0;
cerr << "Finished Loading" << endl;


//Process Frames
int depth = 0;

cerr << "Rotating images about y" << endl;
for (int depth = 0; depth < image_vector[0]->width; depth++) {
IplImage *result = cvCreateImage(cvSize(image_vector.size(),
                          image_vector.back()->height), IPL_DEPTH_32F, 3);

for (int x = 0; x < image_vector.size(); x++) {
for (int y = 0; y < image_vector[x]->height; y++) {
CvScalar pix = cvGet2D(image_vector[x], y, depth);
cvSet2D(result, y, x, pix);
}
}
std::string s;
std::stringstream out;
out<< setw(3) << setfill('0') <<depth;
s = out.str();
s.append(".jpg");
cvSaveImage(s.c_str(), result);
//result_vector.push_back(result);
}
cerr << "Done rotating images about y" << endl;

return (EXIT_SUCCESS);
}