Paper Reading #3: Pen + Touch = New Tools

Reference Information

Pen + Touch = New Tools by Ken Hinkley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton.  Microsofy Research, One Microsoft Way, Redmond WA 98052.  Presented at UIST 2010 on October 3rd - 6th in New York.

Author Bios

• Ken Hinckley is a principal research at Microsoft Research. He holds a PhD in Computer Science from the University of Virginia.His research focus is to enhance input vocabulary that one can express in common user interfaces. 
• Koji Yatani is a PhD candidate at the University of Toronto and previously worked for Microsoft Research in Redmond
• Michel Pahud works for Microsoft Research and holds a PhD in parallel computing from the Swiss Federal Institute of Technology. His work is focused in the area of user experience
• Nicole Coddington worked for Microsoft Research and is now a senior interface designer at HTC. She has a bachelor's degree in visual communication from the University of Florida.
• Jenny Rodenhouse works for Microsoft Research. She is currently in their Xbox division.
• Andy Wilson is a senior researcher at Microsoft Research with a PhD from MIT's Media Lab. He helped found the Surface Computing group.
• Hrvoje Benko is a researcher of adaptive systems and interaction for Microsoft Research. He received his PhD from Columbia University.
• Bill Buxton is a principal researcher for Microsoft Research with three honorary doctorates. He also holds a Bachelor of Music Degree from Queen's University.

Summary
Hypothesis

There have been advancements in technologies which use Pen interface and Touch interface as a means of interaction. This paper presents a novel way to integrate both functions over a single interface.

Methods

The researchers first performed an experiment to observe how people manipulate tools such as paper, pen, and a design surface when creating a craft.  They told participants to put together a paper notebook consisting of clippings pasted into a notebook and observed how the participants used their hands, the pen, and the clippings in a collection of gestures and techniques for efficiency that we don't often think about.  Later in the paper the authors describe how they applied their observations and integrated them into the system to be tried out by testers.  The participants were asked to experiment with the resulting product and provide feedback about its usability.

Results

The results showed that while writing, the secondary tasks of creating new objects, flipping back and forth between pages, and navigating to the extended workspace can all help retrieve, compare, and create task- relevant information.The approach to combine pen and touch interaction was appealing to users. They quickly formed habits around the general pattern (hold touch + gesture with pen) .

Contents

The authors tried to combine pen and touch gestures into a combination they called pen + touch. Most of previous work in this area resulted in unintuitive gestures or used buttons. The authors suggested that pen and touch generally operated in different fields, but also that gestures should be based on how users interact with physical materials. Their main design considerations were input types, if the interface should change based the task, how to assign devices to hands, how to best map inputs, how many hands to use, if simultaneous or sequential actions are needed, when to use ink or command mode, and whether inputs would be simple or phrased together. The authors decided that in general the pen should be ink, and touch should manipulate objects. Users naturally interleave pen and touch, allowing for more actions.

Discussion

The authors sought a more intuitive usage for touch and pen controls and found one. I believe that they achieved this goal. The paper is interesting because it is very easy to see how this system could be adopted almost seamlessly into daily life.  However, I also feel that this isn't particularly novel and could be seen as reinventing the wheel.