Sunday, August 26, 2007


I got an email full of questions this morning and I thought I would post it here along with my response so that it might help others with similar questions. If you can contribute to the answer, please reply. Thanks!

Dear Miketavius,

I'm xxxxxxx, a CS undergraduate student who's planning to work on my
graduation project for my last year .. I've just found these forums n I
checked ur posts and ur blog .. that's GREAT ..
I really hoped if I could ask for some advice ..

We're trying to build a SDK for Multitouch input, we're building the HW, we
will have the sequence of frames captured, process it for blob detection n
tracking, raise events .. then we'll be building a 3D-sculpting tool on top
of it as a demonstration ..

I bought the acrylic sheet, IR-LEDs, n fixed a frame .. I hacked an old
webcam as well ..
I have a few questions regarding the HW:
- In order to see clear blobs, I had to turn of the light in the whole room
.. in order not to do so, I tried putting a magnetic tape instead of the
IR-lens inside the cam, but it's too dark n the blobs are so dim n weak ..
I tried using a camera film, I used 3 layers n they were still too bright
but far more blurry!!!! Suggestions?? I really hoped I wouldn't end up
having to use a totally dark room else it won't work, u know !!!
- Do I have to put milkpaper or anything on top or bellow the acrylic
sheet? Why?

Concerning the SW part, we wanted to do the whole thing .. capturing the
images is not that important so I wouldn't mind using smthing already built
for that purpose ..
But the blob detection n tracking (everything starting from the point where
u only have a sequence of images), we want to do all that ..
We decided to work in C++ ..
Any references? Source code to understand how things work? papers? anything
to read?
We still have one year to go in college so we haven't studied image
processing yet .. we need primitive source .. for a very fresh start ..
I just need some advice or guidance .. we're at the very first steps .. I'm
sure u've been there at some point .. just help us kick-start with any
suggestions ..

The lighting is really the most important aspect of creating a
rear illuminated multitouch screen. I found that using an IR bandpass
filter to match the exact wavelength of the IR leds helps
tremendously. In my case, I have 880nm leds and an 880nm bandpass
filter. This means that my filter's light transmittance peaks at
880nm and falls off sharply after that. Even at the highest point on
the light transmittance curve though, it only lets through 50%-60% of
the light. This means that you will need more light to evenly light
your screen as opposed to using film negatives. While this is a
drawback, using a bandpass filter versus film negatives does help
lessen the effects of ambient light interference. As long as my
screen is not directly facing sunlight or another light, I can use it
in a room with the lights on.
Also, you need to have some sort of diffuser in front of or
behind the acrylic. I've found that blobs show up brighter if you put
the diffuser on the front of the acrylic. Your diffuser can also
serve as your projection surface. In my case I use a single sheet of
vellum paper for both diffuser and projection surface. If you don't
have some kind of milky sheet on the acrylic, then your hand will
reflect back a lot of IR light and it will be more difficult to
recognize individual fingertips.
As far as software goes, I'm glad to hear you're going to program
it yourself. It's been a fun challenge and I've learned a lot so
far... and my project still has a long way to go! For image
processing, I use openCV. There's an add-on to the project that does
blobtracking. I've heard that by using a floodfill based blobtracking
method though that one can achieve faster results than with openCV's
blobtracking. I don't have any references for that, but I'm sure if
you google "floodfill blob recognition" something will come up.

Sunday, August 12, 2007

Calibration test

Here's a vid of me testing out my touch surface and the touch/projection calibration.

Saturday, August 11, 2007

finger tracking update

I've been working on improving the finger tracking of my multitouch software. In the video below, you'll see my fingers with colored lines behind them. When a line disappears, or changes colors, that means the finger has been recognized as a new touch. Obviously there are still some bugs to work out, but overall I think it's coming along well.

Friday, August 10, 2007

Everything is illuminated

So lately I've had trouble illuminating my entire multitouch screen. I have been using a simple IR illuminator I constructed on a breadboard with 16 leds, but that would only light up a circular area in the middle of the screen. I needed something better and more adjustable and here is what I came up with.

I took a small piece of perf board and wired up 24 ir leds, 3 rows of 8 in parallel. I put this perf board in a small radio shack project box. After making two of these, I bent a aluminum bar in a U shape and attached the lights on each side with a nut and bolt so I can adjust them vertically. Finally, I drilled a hole in the bottom of the U and attached it to some floodlamp stands I had laying around. Viola! I made two of the U light assemblies and hooked them both up to the same 12V cpu power supply. They do a nice job of lighting up my entire screen and are easy to adjust. Here's a pic of what one of them look like.

Hopefully this can help somebody out who is struggling with how to build a simple IR illuminator. Oh, and these things were fairly cheap to build, which I find to be very important as my multitouch bill seems to keep growing and growing ;)