Ramsin's Research

SIGGRAPH 2008

2008-08-12T22:21:00.001-07:00

So I went up to SIGGRAPH today to meet up with some of the NUIGroup guys. I was mainly interested in seeing their new tech demos, but I didn't see all that much. There were a few exhibits that caught my attention.

IncreTable is a multi-touch table designed to work with objects on the surface of the table. While it is a multi-touch table, they did not demo any of that technology. They had two demos and the multi-touch demo was not accepted. Instead, they had a ZCam mounted above the table that measured depth to alter terrain in a virtual game world.

The ZCam mounted above the table with a projector alongside

The virtual 3D world and the physical objects placed on the table

They had a second application that used the same setup. What was really interesting was that they explored the reverse approach as well, namely manipulating physical objects with virtual ones. They had a virtual domino setup that when triggered would actually knock over real dominoes placed on the table using a cylindrical bluetooth actuator identified with fiducial markers on the top of it.

The virtual dominoes are on the left and the knocked-over physical dominoes are on the right.

Also, something very interesting was that they were using Anoto pens to control their program. They seem to have found a surface that you can print the Anoto dot patterns onto and which seems to eliminate the need for the silicone compliant surface. I was unable to find out what the material of the surface was because they weren't ready to disclose that information just yet. It seems they spent a year researching it.

Butterfly Haptics was a pretty interesting exhibit. They use magnetic levitation to create joystick input devices that you can use to control a 3D world of blocks. I was really amazed at how I felt like I was actually touching objects of different weights. When I stacked blocks on each other, they actually felt heavier to lift. Apparently I was messing up the magnetic fields somehow because the woman running the demo had to reset it a number of times and finally had to ask me to step back before she could reset it.

ForceTile reminded me a lot of the SLAP widgets we are looking to develop. However, they used a second sensor to measure force on a tile by way the tile deformed when pressure was being applied to it.

The tiles were illuminated with the projector and used to represent control buttons for their demo application.

This demo did not have too much information, but the group had developed a 3D world that the manipulated with the widgets on the multitouch surface (on the right). The widgets were labeled with fiducial markers to identify them to the camera system.

This project seems very relevant to our work. It's the same work that Seth sent out to us a while back (here), but I hadn't seen their results. The group has implemented a pulsed LED array that grabs a frame from the camera when the array is off and another one when the pulse is high. When these two images are subtracted, you are left with an image only containing foreground objects illuminated by the FTIR. Apparently it seems to work very well and the two images on the right illustrate the circuit used to drive the system. It would be interesting for us to try.

The remainder of the pictures are located in my flickr album at:
http://www.flickr.com/photos/21704518@N00/sets/72157606701371183/

Flare

2008-08-04T01:31:00.000-07:00

Flare is a Flash library that allows users to create some very interesting visualizations. Although we've decided that the video analysis software will be written in C++ for numerous reasons, I thought it would be interesting to explore how difficult it would be to develop a timeline visualization for someone who has had no prior experience working in Flash.

I spent a few hours going through the Flare tutorial (which is VERY straightforward). I was impressed at how quickly I could prototype something. After the tutorial, I tried to generate a few timelines of data. While the sample data worked fine, I realized that it would take a good deal of additional coding in order to create a Flash front-end for the video analysis tool. Since all the image processing for multi-touch is happening in C++, I already have access to the tracked blobs.

At least for now, I think I will hold off on Flare, although I am really lured by the attractive visualizations it provides. One good thing about tbeta (or reacTIVision or BBTouch for that matter) is that since it can generate TUIO messages, I can always create an AS3-based front-end later on that will receive these messages and be a separate "plug-in" to the system.

Getting up to speed

2008-08-04T00:45:00.000-07:00

I am still hammering out my main research goals/problems, but I am working on a project to develop a multi-touch technology based workbench for video analysis. In doing so, I hope to unveil some potentially poignant problems to address.

Over the last few days, I was working on getting "tbeta" ported over to OS X. Tbeta is a revamping of touchlib, an open-source library for multi-touch application development created by Nuigroup for capturing touch events. However, the new tbeta is built on openFrameworks so that it is cross-platform, cross-compiler, and introduces a user to create a graphical front-end for their multi-touch software. I have been working with a few of guys at Nuigroup to get tbeta up and running. On Friday, I was finally able to get it to compile in XCode and run. It's still buggy and needs to be cleaned up and optimized, but here is a screenshot of it so far.

First Post

2008-08-04T00:38:00.000-07:00

I thought it would be beneficial for me to document the progress of my research in a public blog so that I could share with interested individuals, get feedback on research directions, and catalog my progress.

Earlier this year, I began working in the DCog-HCI laboratory at UCSD with Professor Jim Hollan. I am working towards pursuing my PhD in Computer Vision/HCI.

More information can be found out about me on my website: http://www.risenparadigm.com

Recently I released a Windows-based mouse driver for multi-touch surfaces built on top of touchlib. If you are interested in that, you may access it here:
http://www.risenparadigm.com/multimouse