— Outlook Shortcuts, 2005
“Humans communicate using speech, gesture, and body motion, yet today’s computers do not use this valuable information. Instead, computers force users to sit at a typewriter keyboard, stare at a TV-like display, and learn an endless set of arcane commands, all of which often leads to frustration, inefficiencies and disuse.”
— Mark Lucente, Aug 1999
Mark Lucente wrote this for a panel organized in 1999 in Los Angeles that included Bill Buxton (then with Wavefront, Inc.) and others. At the time, Mark was working on a project called Dreamspace, pictured below.
Does Dreamspace look familiar? It uses a combination of gestures and voice commands, just like this fantasy UX from 2002 you may have seen:
Instead of inspiring us to find new ways to do astronomy or police work, of course, in 2010 these sparks of inspiration gave rise to this instead:
Which is alright. First we hook the kids, then we conquer the future.
I found Mark’s synopsis of the UX issues of 1999 in the archives of the MIT Media Lab Tangible Media Group. I’m currently working my way through these papers and am overwhelmed by how smart the authors are as well as how far behind I am in trying to understand UX and what can be accomplished with a bit of vision.
You have it right – there is a long history here, including supermarket doors that open as a result of the "gesture" of approaching them with an arm-load of groceries. Mark’s work was really good. But it also grew out of great roots (to its credit, not its detriment). Go back to Richard Bolt’s paper, Put-That-There, from 1980:
http://www.media.mit.edu/speech/papers/1980/bolt_SIGGRAPH80_put-that-there.pdf
and the 1979 video:
http://www.youtube.com/watch?v=RyBEUyEtxQo
I think that you will have a huge smile on your face. One of admiration for such early pioneering work. But I think that knowing what some of the pioneers did helps us all be a bit less satisfied how clever we are, and more aware and appreciatative of the foundation on which we build.
History is great. What is ironic is how little attention we pay to it, given that we supposedly live in the "information age".
Nice post. Nice reminder. Thanks.
Put-That-There is amazing. Thank you for linking it. I can see the outlines of everything hobbyists are trying to do with the Kinect today in that 1979 video.
No one is currently taking advantage of the mic built into the Kinect, though, or the speech recognition capabilities of Win 7. It’s almost as if NUI and gesture based UI is replacing SR instead of complementing it. Perhaps combining SR and gesture effectively will be part of the next phase of Kinect hacking.
Your statement about the lack of historical perspective during the "information age" is right on. When changes are happening so rapidly and affect us so profoundly (it’s become a truism that technology transforms how we look at the world) we have an even greater need for explanations of why these changes are happening and how they are coming about.
Probably the speed of technological change is the main reason that we have so little time for analysis — rather than simply a lack of interest. Blogs and tech websites, hopefully, will provide enough of a trail so future historians can reconstruct what is happening to us today.