Paper reading #3 - Pen + Touch = New Tools

• Paper Title: Pen + Touch = New Tools
• Authors: Ken Hinckley, Koji Yatani, Michel Pahud, Nicole Coddington, Jenny Rodenhouse, Andy Wilson, Hrvoje Benko, and Bill Buxton, all members of the Microsoft Research Team.
• Presented at 23rd annual ACM symposium on User interface software and technology, New York, NY, 2010

http://dl.acm.org/citation.cfm?doid=1866029.1866036

Summary

     The words the authors use to sum up this idea are, "The pen writes, touch manipulates, and the combination of pen+touch yields new tools." This is an idea that Hinckley, et al are putting into practice through their development of Manual Deskterity, an application for the Microsoft Surface (which is essentially a touch-screen table) that allows writing with a pen, touch interface editing, and new tools through a combination of the two interfaces. To come up with how to create the features and how they should be used, they studied a group of people doing arts and crafts to observe their motions while working with different tools. Manual Deskterity has now been developed as, in its basic sense, a scrapbooking application, using common tasks, such as writing, cutting, drawing, using straight edges, stacking items, etc. as well as tasks only available on a computer, such as copy-pasting, zooming, and creating new objects out of thin air. they are not yet ready to to move this idea into production yet, and plan to continue research into how this technology can apply to other office interfaces, such as spreadsheets, word processors, etc.

My View

     Let me put this simply. This is awesome! I'm not a scrapbooker myself, but I know people who are that would love this. I have actually seen something with the touch-screen table interface at Disneyland a few months back, but it didn't use the pen, and thus didn't involve nearly the number of tools that this does. And from what I have seen in the video posted above, all the movements seem very natural and easy to pick up.
     If I were to pick up any faults with this development, I would say that there might be too much stuff going on, and might cause confusion between features. For example, say you wanted to make little dot marks on a picture using the finger-painting technique, but ended up selecting and deselecting the photo in question over and over again. Also, experienced scrapbookers may not view this as a viable alternative to physical items, especially if they already have an existing collection they don't care to transfer.
     Despite these minor faults, I can foresee a vast potential for advancement of this technology. For example, one of the features they showed on the table at Disneyland was that you could place your smartphone on the table, shake it, and all your photos would appear on the table. Paired with Manual Deskterity, you could instantly start editing your photos, and once you're done, upload to Facebook. I could even see this going so far as being used to develop a website, which is scary for me as a web developer, because then I might be out of a job.
     All in all, this is an outstanding application and I cannot wait to see it go out onto the market.