Tag: Kinect

Got an Image Enhancer that can Bitmap?

James Ashley

Every UI platform needs a killer concept.  For the keyboard and mouse it was the Excel sheet.  If you ever watch the rebooted Hawaii Five-0, you’ll realize that for Touch it’s the flick.  Flicking is more satisfying than tapping on sooo many levels.  Birds do it, bees do it, even monkeys in the trees do it.

Gestural interfaces haven’t found that killer concept yet, but it may just be the ability to zoom in on an image.  Like flicking and entering tabular data, killer concepts don’t necessarily have to be clever.  They just have to feel right.

Consider what John Anderton spent his time doing in 2002’s Minority Report.  For the most part, he used innovative fantasy technology (later made real at Oblong Industries) to enhance images on his rather large screen.

Go back even further and you’ll recall Rick Deckard used speech recognition to enhance an image in 1982’s Blade Runner.  This may be the first inkling any of us had of the true purpose of NUI.

It obviously left an impression on the zeitgeist because every movie or TV show attempting to demonstrate technological sophistication on the cheap (CSI being the biggest culprit) managed to insert an “enhance” scene into their franchise somewhere.

And if you happened to have a movie with no budget, there was no reason you should let this stop you.

And while we’re getting nonstalgic for NUI, let’s not forget to give credit where credit is due. Before Leap Motion, before Microsoft’s Kinect, before Oblong’s g-speak, even before Minority Report, there was the NES Power Glove:

And in the decades after, all we’ve managed to do is to enhance that killer concept.

Book Announcement

James Ashley

My colleague Jarrett Webb and I have reached the halfway point on our book for APress this week, so it finally seems safe to announce that we are working on Beginning Kinect Programming with the Kinect SDK to be released towards the end of the year.  My original plan for September and October was to blog feverishly on the Kinect SDK, but given the tyranny of the book schedule, I’m finding it hard not to put everything related to the Kinect into the book instead.  Towards the end of the process, however, we’ll likely have odds and ends that do not make sense in a Beginner book that will, nevertheless, want to be said.  Those odds and ends will eventually make their way here.

In the meantime, the big news is WinRT.  Besides the fact that everyone and their neighbor is already writing their reflections on BUILD, obviating any need for me to contribute, I’ve actually already written a BUILD summary for Razorfish, which should appear on www.emergingexperiences.com shortly.

I can’t resist commenting, all the same, that while I have javascript and C# under my belt, C++ is in my near future.  No love or time for F#, sadly.

“K” is for Hacking

James Ashley3 Comments

K

At the Atlanta .NET User Group presentation on hacking the Kinect last night, I promised to provide instructions on installing the necessary components for hacking the Kinect with the OpenNI stack.  I already feel some trepidation about doing this as figuring out these minimal first steps has always been a bit of an initiation rite for the various members of the OpenNI forums.  Strangely enough, there are few straight-forward walkthroughs on installing basic components.

The few that can be found on the web typically leave out key steps or provide the steps out of order — almost as if there is a general conspiracy to keep the details of these essential steps secret.

You are going to install three pieces: 1. SensorKinect which is the drivers that allow communication between the Kinect device and your PC 2. OpenNI which is an API that sits on top of the drivers and 3. NITE which is a higher level API that sits on top of OpenNI.  4. Additionally, you are going to copy some necessary config files onto your machine.

1. First, download the OpenNI component.  It can be found here.  You can choose to download from either the stable or the unstable branch.  The latter will provide more functionality while the former will be … well … more stable.

It may seem strange that you will be installing the higher level API before you install the drivers they sit on, but that’s the way it has to be done.  To do this in any other order risks opening a rift in space-time and ending the world as we know it.

Install OPENNI-Win32-1.0.0.25.exe (or whatever version happens to be current at the time of this reading).

2. Next you want SensorKinect.  You will need to create a github account to download it from here. 

Install SensorKinect-Win32-5.0.0.exe (or whatever version happens to be current at the time of this reading).

3. Download the NITE middleware which, like OpenNI, comes in both stable and unstable flavors. 

Install NITE-Bin-Win32-v1.3.0.18.exe (or whatever version happens to be current at the time of this reading).  Use the following key: 0KOIk2JeIBYClPWVnMoRKn5cdY4= when prompted.

4. Download and extract the KinectXMLs.zip file: http://box5921.temp.domains/~imagipi2/codesamples/kinectxmls.zip

Copy KinectXMLs\OpenNI\SamplesConfig.xml to C:\Program Files (x86)\OpenNI\Data\

Copy NITE\Sample-Scene.xml, NITE\Sample-Tracking.xml and NITE\Sample-User.xml to C:\Program Files (x86)\Prime Sense\NITE\Data\

These instructions were passed on to me by my colleague Alex Nichols who keeps them in a safe deposit box.  He, in turn, copied these instructions by candlelight from Steven Dawson, a shadowy and rarely seen uber-developer known for writing kinect hacks like this: http://www.engadget.com/2010/12/05/razorfish-ports-davinci-interface-to-kinect-makes-physics-cool/

You know you have installed everything correctly if a portal to another dimension does not open up above your Kinect.  Just to make sure, though, browse to C:\Program Files (x86)\OpenNI\Samples\Bin\Release (“Program Files” if you are on a 32-bit OS).  Plug your Kinect into a USB port and run NiViewer.exe.  If you can see yourself in the screen that pops up, then the OpenNI layer is working correctly.

Next, browse to C:\Program Files (x86)\Prime Sense\NITE\Samples\Bin\Release.  Try running Sample-PointViewer.exe.  You should quickly know if your NITE middleware is installed correctly.

In my experience, depending on which code branch you install, various samples may not work.  That’s okay.  As long as some of them work everything is fine.

Now you’ll want to try programming something.  Kinect hacker Vangos Pterneas provides several excellent examples you can peruse:

http://www.studentguru.gr/blogs/vangos/archive/2011/01/28/kinect-and-wpf-getting-the-raw-and-depth-image-using-openni.aspx

http://www.studentguru.gr/blogs/vangos/archive/2011/02/12/kinect-and-wpf-painting-with-kinect-using-openni.aspx

http://www.studentguru.gr/blogs/vangos/archive/2011/03/15/kinect-and-wpf-complete-body-tracking.aspx

Again, sometimes these work and sometimes they don’t.  If they aren’t working for you don’t worry too much about it.  Just move on.

Next, you’ll want to play with this Visual Studio 2010 C++ project on github: https://github.com/OpenNI/SampleAppSinbad  It is a 3D model of a piratical ogre that will track your movements.  You have to do some configuration if you want to run it from VS2010.  You can, however, just run the executable in the bin directory.  Reach behind your head with both hands to play with the swords.

I also showed several videos last night providing a visual history of gestural interfaces from laboratories, to the movies, to the mass-market, and finally to the hacking community.  If you are a fan of fantasy-UX, then all of these should already be familiar to you:

Star Trek Voyage Home (1986): http://www.youtube.com/watch?v=v9kTVZiJ3Uc&feature=related

Put That There (1979): http://www.youtube.com/watch?v=RyBEUyEtxQo

DreamSpace (1998): http://www.research.ibm.com/natural/dreamspace/#example

Minority Report (2002): http://www.youtube.com/watch?v=NwVBzx0LMNQ

Iron Man (2008): http://www.youtube.com/watch?v=PYjlMflhysc

Project Natal: http://www.youtube.com/watch?v=Mf44bWQr3jc

12 Best Kinect Hacks: http://www.youtube.com/watch?v=ho8KVOe_y08

Again, to everyone who came out for last night’s presentation — you were extremely patient and I can’t thank you enough for your time and your enthusiasm.