Monday, December 3, 2007

finished screen

I finished my projection screen. It was a very messy endeavor, but I'm pleased with the results. The screen is composed of a front layer of rosco grey rear projection screen adhered to a piece of 1/4" acrylic. The image really looks great, the images below don't do them justice. Anyways, that's all for today I think... no more updates until I get all the parts in to make my new lights.

Friday, November 30, 2007

Say Cheese!

I've been testing out the Canon SDK for an powershot model I have and I finally got it to work! I made a simple console app that starts the sdk, connects to the camera, and then takes a pictures and saves it to the computer. Not bad for a few hours of shuffling through documentation ;) How this works into my multitouch project... that's my secret for now!

In other news, I made the final screen for my multitouch box using the glue method I previously described. I'll admit, it was much more challenging than I initially thought, simply because of the size. I ended up diluting the glue to a thin consistency and then spraying it on the acrylic. I then used a large squeegee to get all the excess glue and air out from under the rosco grey. Overall, the screen looks great. I should be able to get it reinstalled into the big box so I can test the image quality.

Tuesday, November 20, 2007


I'll be in the garage for the next few days! See-ya Blogosphere :)

Sunday, November 18, 2007

Photo update

In my last post I alluded to my current project, which is a big multitouch box. Here's a picture of what it looks like (with one of the side panels off). I was doing a projection test with a DVD of the movie Robots.

So anyways, nothing that impressive yet. I had trouble getting the rosco to stretch tight over my acrylic, so I had to figure out a way to adhere it to the surface. I tried various spray adhesives from 3M and I found that while many offer a strong bond, they all leave a visible texture.

I came up with a solution that seems to work quite well. Naturally, it was the most simplistic solution smile The secret adhesive is… dun dun dun… Elmer’s glue. That’s right… the white stuff we got all over our hands and clothes in grade-school.

I diluted some white glue with hot water so it was thin enough to be brushed onto the acrylic. The trick is to work with the rosco as if you are applying some window film. Apply a liberal amount of glue to the acrylic and then lay down the rosco starting from one end (try to minimize air pockets). Next, wet the surface (you don’t have to do this… but just to be nice to the rosco you should) and squeegee the air pockets out. I let mine dry for a few hours and it left me with a perfectly flat rosco/acrylic combo. Projecting onto it, there is no visible pattern or texture like I had with various brands of spray glue.

If this experience has taught me anything… it is to always try the low-tech solution first. That's all for now... more updates to come since I have this next week off from school!

Friday, October 26, 2007

Still Alive!

Here's what I have been up to lately.

I've moved on to building a larger, more polished prototype. I am creating a mobile multitouch "wall" that measures approx 6' tall, 64" wide, 30" deep. The actual screen area will be roughly 54"x32".

So far construction is going well. I'll post pictures as soon as I can!

Friday, September 28, 2007

Busy busy!

I have been very busy with school lately so I haven't worked much on the software side of things. Managed to get my software running at around 70fps w/ the camera capturing at 60 :) Things run REALLY smooth now... even more so than in my videos. I'll post a video sometime in the future once I get the wall up and running. Figuring out how to build a 36"x64" wall that can move around is fairly difficult!

Monday, September 3, 2007

Squash those bugs!

If you saw the video of my photo app in the last post, you'll notice some flickering when I touch two photos that overlap. I fixed that bug (it was a problem with ordering). I also added velocities to the photos, so you can flick 'em and send them sliding down the screen. It's fun :)

Saturday, September 1, 2007

Photo App Video!

So, I've been working on the basecode for my multitouch setup for a while now. I made this simple photo app as a test. It still needs a little work, but I'm pretty pleased with it so far. Take a look!

Sunday, August 26, 2007


I got an email full of questions this morning and I thought I would post it here along with my response so that it might help others with similar questions. If you can contribute to the answer, please reply. Thanks!

Dear Miketavius,

I'm xxxxxxx, a CS undergraduate student who's planning to work on my
graduation project for my last year .. I've just found these forums n I
checked ur posts and ur blog .. that's GREAT ..
I really hoped if I could ask for some advice ..

We're trying to build a SDK for Multitouch input, we're building the HW, we
will have the sequence of frames captured, process it for blob detection n
tracking, raise events .. then we'll be building a 3D-sculpting tool on top
of it as a demonstration ..

I bought the acrylic sheet, IR-LEDs, n fixed a frame .. I hacked an old
webcam as well ..
I have a few questions regarding the HW:
- In order to see clear blobs, I had to turn of the light in the whole room
.. in order not to do so, I tried putting a magnetic tape instead of the
IR-lens inside the cam, but it's too dark n the blobs are so dim n weak ..
I tried using a camera film, I used 3 layers n they were still too bright
but far more blurry!!!! Suggestions?? I really hoped I wouldn't end up
having to use a totally dark room else it won't work, u know !!!
- Do I have to put milkpaper or anything on top or bellow the acrylic
sheet? Why?

Concerning the SW part, we wanted to do the whole thing .. capturing the
images is not that important so I wouldn't mind using smthing already built
for that purpose ..
But the blob detection n tracking (everything starting from the point where
u only have a sequence of images), we want to do all that ..
We decided to work in C++ ..
Any references? Source code to understand how things work? papers? anything
to read?
We still have one year to go in college so we haven't studied image
processing yet .. we need primitive source .. for a very fresh start ..
I just need some advice or guidance .. we're at the very first steps .. I'm
sure u've been there at some point .. just help us kick-start with any
suggestions ..

The lighting is really the most important aspect of creating a
rear illuminated multitouch screen. I found that using an IR bandpass
filter to match the exact wavelength of the IR leds helps
tremendously. In my case, I have 880nm leds and an 880nm bandpass
filter. This means that my filter's light transmittance peaks at
880nm and falls off sharply after that. Even at the highest point on
the light transmittance curve though, it only lets through 50%-60% of
the light. This means that you will need more light to evenly light
your screen as opposed to using film negatives. While this is a
drawback, using a bandpass filter versus film negatives does help
lessen the effects of ambient light interference. As long as my
screen is not directly facing sunlight or another light, I can use it
in a room with the lights on.
Also, you need to have some sort of diffuser in front of or
behind the acrylic. I've found that blobs show up brighter if you put
the diffuser on the front of the acrylic. Your diffuser can also
serve as your projection surface. In my case I use a single sheet of
vellum paper for both diffuser and projection surface. If you don't
have some kind of milky sheet on the acrylic, then your hand will
reflect back a lot of IR light and it will be more difficult to
recognize individual fingertips.
As far as software goes, I'm glad to hear you're going to program
it yourself. It's been a fun challenge and I've learned a lot so
far... and my project still has a long way to go! For image
processing, I use openCV. There's an add-on to the project that does
blobtracking. I've heard that by using a floodfill based blobtracking
method though that one can achieve faster results than with openCV's
blobtracking. I don't have any references for that, but I'm sure if
you google "floodfill blob recognition" something will come up.

Sunday, August 12, 2007

Calibration test

Here's a vid of me testing out my touch surface and the touch/projection calibration.

Saturday, August 11, 2007

finger tracking update

I've been working on improving the finger tracking of my multitouch software. In the video below, you'll see my fingers with colored lines behind them. When a line disappears, or changes colors, that means the finger has been recognized as a new touch. Obviously there are still some bugs to work out, but overall I think it's coming along well.

Friday, August 10, 2007

Everything is illuminated

So lately I've had trouble illuminating my entire multitouch screen. I have been using a simple IR illuminator I constructed on a breadboard with 16 leds, but that would only light up a circular area in the middle of the screen. I needed something better and more adjustable and here is what I came up with.

I took a small piece of perf board and wired up 24 ir leds, 3 rows of 8 in parallel. I put this perf board in a small radio shack project box. After making two of these, I bent a aluminum bar in a U shape and attached the lights on each side with a nut and bolt so I can adjust them vertically. Finally, I drilled a hole in the bottom of the U and attached it to some floodlamp stands I had laying around. Viola! I made two of the U light assemblies and hooked them both up to the same 12V cpu power supply. They do a nice job of lighting up my entire screen and are easy to adjust. Here's a pic of what one of them look like.

Hopefully this can help somebody out who is struggling with how to build a simple IR illuminator. Oh, and these things were fairly cheap to build, which I find to be very important as my multitouch bill seems to keep growing and growing ;)

Monday, July 30, 2007

Progress :)

It's good to be finally making some progress. I've continued working on my multitouch project. In the past week or so I've been very focused on getting my new camera to work. I made a neat assembly that combined my new webcam lens and my IR bandpass filter from Thorlabs. The cool part is that I made it out of a bottle cap :)

Also, on the software side of things, I just got the blob tracking working. Nothing cool to show yet as it still needs to be refined, but this was a major hurdle on the way to programming some real multitouch apps. More stuff to come in the near future...

Friday, July 13, 2007

Smoke and Mirrors

Well I got my projector today! I took over the spare bedroom in the house and setup a makeshift table to test things out. I wanted to check for a few things...

a) how difficult would it be to get the projector image straight when going from horizontal to vertical via the mirror reflection

b) how much infrared light the projector gave off

c) if the projector would look washed out with some regular lights on

Here's what I found out. It was pretty easy to get the projector to show up on my screen. I was even more surprised with how well vellum worked as a projection surface. When I turned on my IR webcam to check the projector it picked up virtually no IR light. It gets even better though, because even with with all the lights on in the room the projection was still crisp and the contrast looked great. All in all... I'm feeling pretty good about the project. Check out some pics!

Friday, July 6, 2007

More waiting

Well after my last post I wanted to try illuminating my surface from behind with a few IR LEDs. I found that with everything I tried, the LEDs just put way too much focused light on the surface. After talking with cerupcat on the nuigroup forums, he mentioned the use of the GamFusion material from as a diffuser of the IR light. The stuff is pretty cheap so I ordered some along with some more LEDs.

On a more exciting note, I came into some unexpected funding for my project. I can finally afford to buy a projector :)

Wednesday, July 4, 2007

Change of Direction

Well a few days ago I recieved my silicone-rubber sheet from Gravano of After getting it setup on my table and doing some more work on my software I was able to get pretty good finger tracking. The problems came the next day, when my software went crazy because of a little bit of ambient daylight in the room. Since I want to use this technology someday in the classroom, this presents a roadblock.

So, I've now decided to shift my focus from FTIR (sides of acrylic lit with IR LEDs) to diffused illumination (front or rear lit with IR LEDs). So far the results are promising. I've uploaded a video of me testing out the surface.

I think this is definately the way to go. Just looking at the video, it's clear to see I get fairly accurate results using only ambient light. With a rear lit setup, I'm sure I can make this work even better. That's it for today.

Oh, and Happy Birthday Nation... I got you a giftcard.

Wednesday, June 13, 2007

My bubbles!

The past few days have been spent waiting for my silicone rubber compound to arrive in the mail. It came yesterday and so I promptly mixed everything together and poured an approx 1mm layer on top of my acrylic sheet.

Although the surface came out very smooth there are tons of small bubbles within the mixture. The number of bubbles lowers the translucency of the surface and introduces a great deal of noise into the captured image. I'm glad I gave this a try myself as it was a great learning experience... but now I think I'm going to order a sheet from someone who has real experience at making them (Gravano @ NUIGROUP).

So, I'm back to waiting.

Friday, June 8, 2007


I realized I showed a screenshot of my software but didn't bother to show the actual prototype. Here it is, in all it's DIY glory...

Here we go!

Well I'm starting this new blog today. I figured it would be nice to keep track of my current / future projects. A few days ago Microsoft announced their new Surface computing platform. It's basically a coffee table with a multitouch screen on top. The moment I saw it I knew I had to make one... so this brings me to my new project.

I want to build a multitouch table for use in the classroom. I think this kind of technology could really break down the one student per computer barrier we see in classrooms today. A multitouch table would allow students to use technology cooperatively in a way that, currently, is an impossibility for most students. The best part is that the technology the table is based on, Frustrated Total Internal Reflection (FTIR), is fairly cheap to produce and relies mostly on common PC components and other readily available materials. Take a look at this page to get the gist of it.

With the help of the NUIGROUP community, I found enough information to construct a rough prototype screen. So far it does simple blob tracking when I press my fingers on the screen. In a few days the materials for my complaint surface should arrive and I can begin the next phase of my research.