Paper Reading #13: Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces

Reference Information
Combining Multiple Depth Cameras and Projectors for Interactions On, Above, and Between Surfaces
Andrew D. Wilson and Hrvoje Benko
Presented at UIST'10, October 3-6, 2010, New York, New York, USA

Author Bios
Andrew D. Wilson is a senior researcher at Microsoft Research. He was a Cornell Undergrad and earned his Masters and PhD from the MIT Media Lab.
Hrovje Benko has a PhD from Columbia University, he is a researcher at Microsoft Researc.

Summary
Hypothesis
Is LightSpace a usable and better way to interact with applications in 3D space?

Methods
LightSpace was tested at a convention where it was made available for public use. Several hundred people tested and used the system and were encouraged to play and explore the capabilities of LightSpace. The interactions were observed by the researchers in order to obtain usability data and insight into how to make the system more functional and intuitive.

Results
The researchers found that the effective maximum for the system was  six people at a time, however to operate smoothly it was best to keep it at three or below. The reasons for setting limits on users is that the more users that are present the more points the system has to keep track of and the higher the likelihood that users block each others movements from the camera. Users also discovered new ways to use the system that hadn't been done before.

Contents
LightSpace uses multiple projectors and depth cameras to created a simulated environment and allow for multiple interactions.The researchers chose to use their equipment to track, study, and use 4 main interactions: simulated interactive surfaces, through-body transitions between surfaces, picking up objects, and spatial menus.


Discussion
The LightSpace system is an amazing ideas and something that has been the dream of science fiction fans ever since the holodeck was featured on Star Trek. Advances in technology similar to LightSpace could one day make these fantasies a reality. The system that these researchers developed is a great step in that direction for a couple different reasons but the most important is that it takes a great idea and improves upon it by making it portable. The ability for this to be possibly wheeled into lecture hall at the beginning of a day, calibrated, used for various classes and lectures, and then taken to a different location the next day is a big boon for this system if it is ever commercialized.

Paper Reading #12: Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection

Reference Information

Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection

Li-Wei Chan, Hsiang-Tao Wu, Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin, Mike Y. Chen, Jane Hsu, Yi-Ping Hung

Presented at UIST'10, October 3-6, 2010, New York, New York, USA

Author Bios

• Li-Wei Chan is a PhD student at the National Taiwan University and researches with the Image and Vision Lab and iAgent.

• Hsiang-Tao Wu is now a researcher at Microsoft Research Asia but previously attended the National Taiwan University.

• Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin were students at the National Taiwan University during the time of this paper.

• Mike Y. Chen is an assistant professor at the National Taiwan University specializing in human-computer interaction and mobile computing.

• Jane Hsu is a professor at the National Taiwan University and focuses on multi-agent systems and web technologies.

• Yi-Ping Hung is a professor at the National Taiwan University and researches human-computer interaction among other things.

Summary

Hypothesis

Current touchscreen interfaces and devices are limited in size and, therefore, in content.  The authors believe that a solution lies in creating an interface with a larger surface area and using feet as the interaction agents.

Methods

The first user study was done among 16 right footed individuals to find the capabilities and the limitations of the foot based interaction.They examined foot gestures using a target selection task that required participants to select discrete targets along three axes of foot rotation.M-Series Vicon Motion Capture cameras were used to accurately capture and log the movement of a participant’s foot. The selection and confirmation of targets presented on a laptop worked with the click and release of the mouse.

A different set of study among 6 right footed participants used machine learning to identify foot gestures.Based on the design implications from the first study,they focused specifically on plantar flexion and heel rotation gestures. Bayes theorem was used to perform classification of users foot movement.For each gesture type, each target was randomly presented 10 times for training and 50 times for testing. In total, each participant completed 100 gestures in the practice phase and 500 gestures in the testing phase. They wore jeans with pockets to hold iPhones.Four classification tests: across all gestures, across all gesture types, across all gestures in Heel rotation, and across all gestures in plantar flexion were conducted.

Results

There were complaints from the users that only the bottom portion of the building could seen. It was also noted that they wanted to be able to see the upper parts of the buildings as well. This was solved by lifting imView up to see the upper part of the structure. Another problem was the fact that the imView got lost above the table system. Some of the users also expressed their desire to be able to drag the map using the imFlashlight as well. Some of the issues with the imFlashlight was that it was difficult to for the users to be aware of the actual scene. Overall however the participants gave a positive response about the other products such as imLamp.

Contents

The authors developed a programmable infrared tabletop system that allows mobile devices to interact with displayed surfaces through invisible markers. The system allows for on-surface and above-surface interaction and is based on a Direct-Illumination system. Two IR cameras detect finger touches. A standard DLP projector was converted to IR, with touch-glass placed under the surface's diffuser layer. Augmented reality markers change in size to match the mobile camera, which communicates with the system to update marker layouts and requires at least four markers at a time for calibration. Priority goes to the closest camera. Kalman filtering reduces jitteriness. The normal system of touch detection by subtracting the background is ineffective with the changing IR background, so the system instead simulates the background at each frame and then projects expected white areas for a maximum delay of one frame. The simulation stitches together the marker locations. Foregrounds are extracted through thresholds. Making the markers appear as black pixels provides to little illumination, so the authors adjusted the intensity of the black. Camera and projector synchronization keeps several previous frames in memory for the subtraction method. An additional camera is used only for calibration. The four points used for calibration allow for additional devices without extra input. Content is generated in Flash and warped through DirectX.

Discussion

I believe that the authors managed to achieve the goals outlined, as far as researching people's interactions with their feet and applying it to touch technology.