Sunday, August 26, 2007


I got an email full of questions this morning and I thought I would post it here along with my response so that it might help others with similar questions. If you can contribute to the answer, please reply. Thanks!

Dear Miketavius,

I'm xxxxxxx, a CS undergraduate student who's planning to work on my
graduation project for my last year .. I've just found these forums n I
checked ur posts and ur blog .. that's GREAT ..
I really hoped if I could ask for some advice ..

We're trying to build a SDK for Multitouch input, we're building the HW, we
will have the sequence of frames captured, process it for blob detection n
tracking, raise events .. then we'll be building a 3D-sculpting tool on top
of it as a demonstration ..

I bought the acrylic sheet, IR-LEDs, n fixed a frame .. I hacked an old
webcam as well ..
I have a few questions regarding the HW:
- In order to see clear blobs, I had to turn of the light in the whole room
.. in order not to do so, I tried putting a magnetic tape instead of the
IR-lens inside the cam, but it's too dark n the blobs are so dim n weak ..
I tried using a camera film, I used 3 layers n they were still too bright
but far more blurry!!!! Suggestions?? I really hoped I wouldn't end up
having to use a totally dark room else it won't work, u know !!!
- Do I have to put milkpaper or anything on top or bellow the acrylic
sheet? Why?

Concerning the SW part, we wanted to do the whole thing .. capturing the
images is not that important so I wouldn't mind using smthing already built
for that purpose ..
But the blob detection n tracking (everything starting from the point where
u only have a sequence of images), we want to do all that ..
We decided to work in C++ ..
Any references? Source code to understand how things work? papers? anything
to read?
We still have one year to go in college so we haven't studied image
processing yet .. we need primitive source .. for a very fresh start ..
I just need some advice or guidance .. we're at the very first steps .. I'm
sure u've been there at some point .. just help us kick-start with any
suggestions ..

The lighting is really the most important aspect of creating a
rear illuminated multitouch screen. I found that using an IR bandpass
filter to match the exact wavelength of the IR leds helps
tremendously. In my case, I have 880nm leds and an 880nm bandpass
filter. This means that my filter's light transmittance peaks at
880nm and falls off sharply after that. Even at the highest point on
the light transmittance curve though, it only lets through 50%-60% of
the light. This means that you will need more light to evenly light
your screen as opposed to using film negatives. While this is a
drawback, using a bandpass filter versus film negatives does help
lessen the effects of ambient light interference. As long as my
screen is not directly facing sunlight or another light, I can use it
in a room with the lights on.
Also, you need to have some sort of diffuser in front of or
behind the acrylic. I've found that blobs show up brighter if you put
the diffuser on the front of the acrylic. Your diffuser can also
serve as your projection surface. In my case I use a single sheet of
vellum paper for both diffuser and projection surface. If you don't
have some kind of milky sheet on the acrylic, then your hand will
reflect back a lot of IR light and it will be more difficult to
recognize individual fingertips.
As far as software goes, I'm glad to hear you're going to program
it yourself. It's been a fun challenge and I've learned a lot so
far... and my project still has a long way to go! For image
processing, I use openCV. There's an add-on to the project that does
blobtracking. I've heard that by using a floodfill based blobtracking
method though that one can achieve faster results than with openCV's
blobtracking. I don't have any references for that, but I'm sure if
you google "floodfill blob recognition" something will come up.


nora said...

hey mikey do you have an email address?
I was playing around with my illuminator today but I cnat get blobs to show up -
I was just wondering was it something to do with my setup?

Mikey said...

Hey Nora,
My email address is mtunderwood AT gmail DOT COM.

Martin said...

I'm assembling a DI setup as well. Just a clarification question. You write 'diffuser on the front of the acrylic'. Do you mean you put vellum paper on the camera-side of the acrylic or the finger-side?
Thanks for the knowledge,

Mikey said...

When I write "front", it's always in reference to the side the user will touch. Hope that helps to clear things up.