Paper Reading #12: Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection

Reference Information

Enabling Beyond-Surface Interactions for Interactive Surface with An Invisible Projection

Li-Wei Chan, Hsiang-Tao Wu, Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin, Mike Y. Chen, Jane Hsu, Yi-Ping Hung

Presented at UIST'10, October 3-6, 2010, New York, New York, USA

Author Bios

• Li-Wei Chan is a PhD student at the National Taiwan University and researches with the Image and Vision Lab and iAgent.

• Hsiang-Tao Wu is now a researcher at Microsoft Research Asia but previously attended the National Taiwan University.

• Hui-Shan Kao, Ju-Chun Ko, Home-Ru Lin were students at the National Taiwan University during the time of this paper.

• Mike Y. Chen is an assistant professor at the National Taiwan University specializing in human-computer interaction and mobile computing.

• Jane Hsu is a professor at the National Taiwan University and focuses on multi-agent systems and web technologies.

• Yi-Ping Hung is a professor at the National Taiwan University and researches human-computer interaction among other things.

Summary

Hypothesis

Current touchscreen interfaces and devices are limited in size and, therefore, in content.  The authors believe that a solution lies in creating an interface with a larger surface area and using feet as the interaction agents.

Methods

The first user study was done among 16 right footed individuals to find the capabilities and the limitations of the foot based interaction.They examined foot gestures using a target selection task that required participants to select discrete targets along three axes of foot rotation.M-Series Vicon Motion Capture cameras were used to accurately capture and log the movement of a participant’s foot. The selection and confirmation of targets presented on a laptop worked with the click and release of the mouse.

A different set of study among 6 right footed participants used machine learning to identify foot gestures.Based on the design implications from the first study,they focused specifically on plantar flexion and heel rotation gestures. Bayes theorem was used to perform classification of users foot movement.For each gesture type, each target was randomly presented 10 times for training and 50 times for testing. In total, each participant completed 100 gestures in the practice phase and 500 gestures in the testing phase. They wore jeans with pockets to hold iPhones.Four classification tests: across all gestures, across all gesture types, across all gestures in Heel rotation, and across all gestures in plantar flexion were conducted.

Results

There were complaints from the users that only the bottom portion of the building could seen. It was also noted that they wanted to be able to see the upper parts of the buildings as well. This was solved by lifting imView up to see the upper part of the structure. Another problem was the fact that the imView got lost above the table system. Some of the users also expressed their desire to be able to drag the map using the imFlashlight as well. Some of the issues with the imFlashlight was that it was difficult to for the users to be aware of the actual scene. Overall however the participants gave a positive response about the other products such as imLamp.

Contents

The authors developed a programmable infrared tabletop system that allows mobile devices to interact with displayed surfaces through invisible markers. The system allows for on-surface and above-surface interaction and is based on a Direct-Illumination system. Two IR cameras detect finger touches. A standard DLP projector was converted to IR, with touch-glass placed under the surface's diffuser layer. Augmented reality markers change in size to match the mobile camera, which communicates with the system to update marker layouts and requires at least four markers at a time for calibration. Priority goes to the closest camera. Kalman filtering reduces jitteriness. The normal system of touch detection by subtracting the background is ineffective with the changing IR background, so the system instead simulates the background at each frame and then projects expected white areas for a maximum delay of one frame. The simulation stitches together the marker locations. Foregrounds are extracted through thresholds. Making the markers appear as black pixels provides to little illumination, so the authors adjusted the intensity of the black. Camera and projector synchronization keeps several previous frames in memory for the subtraction method. An additional camera is used only for calibration. The four points used for calibration allow for additional devices without extra input. Content is generated in Flash and warped through DirectX.

Discussion

I believe that the authors managed to achieve the goals outlined, as far as researching people's interactions with their feet and applying it to touch technology.