Today is the last day to innovate before tomorrow …

[This will be the last post before the Mayan apocalypse tomorrow.]

There have already been some very interesting blog posts on other sites predicting the trajectory of technology in 2013.  Worthy of special mention is this excellent overview from Frog Design as well as this one from PSFK.

An interesting feature of all these predictions is that they are an amalgamation of current business trends and futuristic American movies.  Sci-fi movies provide a direction while business (especially retail) provides the funding.  Think of it as a sort of merchandise-celuloidal complex creating our collective future.

The central flaw of practically all the predictions linked above is that they are heavily influenced by American science fiction.  American science fiction, however, is a mere shadow of and several decades behind Japanese science fiction.  I want to correct that today by basing my 2013 Technology Trends predictions on the advanced research occurring in the Japanese futuristic anime industry.

johnny9

1. Giant Robots – 2013 will finally see the arrival of giant robots.  These should more properly be thought of as Gundam or giant suits of armor rather than robots (in the US our pre-occupation with robotics has seriously undermined our edge in this technological frontier) but for the sake of brevity I’ll continue to refer to them as robots for now.

Suidobashi Heavy Industries put their first Mech up for sale earlier this year (youtube link).  Over the next year, we can expect to see giant robots only getting bigger and dropping in price as they go into mass production. 

You should definitely trade in your Prius for one of these rugged commuter vehicles.  Not only will you be able to walk right over most commuter traffic, but you’ll also find your daily commute is much more enjoyable and comfortable as the anti-grav features kick in.  Giant Robots are also good for settling disputes with your neighbors and with your home owner’s association.  Even in rest mode, they become interesting conversation pieces when placed on your front lawn.

You can see a future vision video (much like Google’s vision video for Project Glass) on how giant robots will be used in the near future here.

stargate

2. Wormholes – Created by a race of aliens known as The Ancients, the wormhole travel system was discovered by the US Airforce about fifteen years ago and will be declassified and integrated by the TSA into commercial aviation routes in 2013.  Layovers on Beta Pictoris b and Kepler-42c are imminent.

walkingdead

3. Zombies – The US Cloning program will face a setback in 2013.  For the past five years, all major political figures as well as Hollywood A-List celebrities have been cloned in order to assure the smooth transition of power in government and entertainment.  Have you ever wondered how George Clooney stays so young?  Cloning.

In 2013, however, impurities introduced into the manufacture of clones (currently managed by the Umbrella Corporation) will turn clones of US House members into voracious and infectious brain eaters.  The US Congress will quickly turn the American populous into a rabid, ugly and mindless horde incapable of rational thought and obeying only raw emotions and appetites.

Only those who never leave their homes or watch cable news will be safe.

IMG_1254

4. Tablets – I think tablets are going to be really big in 2013.  Over the past several years I’ve noticed a subtle trend in which cameras have been flattened out and had phone-calling capabilities added to them.  Why phone companies rather than camera companies are driving this is a mystery to me, but more power to them.  Between 2010 and today these cameras have been getting bigger and bigger and are now even touch-enabled!  In 2013, I predict the arrival of 22”, 32” and even 55” touch-enabled cameras called “tablets” that people can comfortably carry around with them in their cars (or in their giant robots).  These tablets can even double as mirrors or flashlights!

3Gear Systems Kinect Handtracking API Unwrapping

WP_000542

I’ve been spending this last week setting up the rig for the beta hand detection API recently published by 3Gear Systems.  There’s a bit of hardware required to position the two Kinects correctly so they face down at a 45 degree angle.  The Kinect mounts from Amazon arrived within a day and were $6 each with free shipping since I never remember to cancel my Prime membership.  The aluminum parts from 80/20 were a bit more expensive but came to just a little above $100 with shipping.  We already have lots of Kinects around the Razorfish Emerging Experiences Lab, so that wasn’t a problem.

WP_000543

80/20 surprisingly doesn’t offer a lot of instruction on how to put the parts of the aluminum frame together so it took me about half-an-hour of trial-and-error to figure it out.  Then I found this PDF explaining what the frame should end up looking like deep-linked on the 3Gear website and had to adjust the frame to get the dimensions correct.

WP_000544

I wanted to use the Kinect for Windows SDK and, after some initial mistakes, realized that I needed to hook up our K4W Kinects rather than the Kinect for Xbox Kinects to do that.  When using OpenNI rather than K4W (the SDK supports either) you can use either the Xbox Kinect or the Xtion sensor.

My next problem was that although the machine we were building on has two USB Controllers, one of them wasn’t working, so I took a trip to Fry’s and got a new PCI-E USB Controller which ended up not working.  So on the way home I tracked down a USB Controller from a brand I recognized, US Robotics, and tried again the next day.  Success at last!

WP_000545

Next I started going through the setup and calibration steps here.  It’s quite a bit of command line voodoo magic and requires very careful attention to the installation instructions – for instance, install the C++ redistributable and Java SE.

WP_000546

After getting all the right software installed I began the calibration process.  A paper printout of the checkerboard pattern worked fine.  It turns out that the software for adjusting the angle of the Kinect sensor doesn’t work if the sensor is on its side facing down so I had to click-click-click adjust it manually.  That’s always a bit of a scary sound.

WP_000554

Pretty soon I was up and running with a point cloud visualization of my hands.  The performance is extremely good and the rush from watching everything working is incredible.

WP_000556

Of the basic samples, the rotation_trainer programmer is probably the most cool.  It allows one to rotate a 3D model around the Y-axis as well as around the X-axis.  Just this little sample opens up a lot of cool possibilities for HCI design.

WP_000557

From there my colleagues and I moved on to the C++ samples.  According to Chris Twigg from 3Gear, this 3D chess game (with 3D physics) was written by one of their summer interns.  If an intern can do this in a month … you get the picture.

I’m fortunate to get to do a lot of R&D in my job at Razorfish – as do my colleagues.  We’ve got home automation parts, arduino bits, electronic textiles, endless Kinects, 3D walls, transparent screens, video walls, and all manner of high tech toys around our lab.  Despite all that, playing with the 3Gear software has been the first time in a long time that we have had that great sense of “gee-whiz, we didn’t know that this was really possible.”

Thanks, 3Gear, for making our week!

Two Years of Kinect

As we approach the second anniversary of the release of the Kinect sensor, it seems appropriate to take inventory of how far we have come. Over the past two months, I have had the privilege of being introduced to several Kinect-based tools and demos that exemplify the potential of the Kinect and provide an indication of where the technology is headed.

restOnDesk

One of my favorites is a startup in San Francisco called 3Gear Systems. 3Gear have conquered the problem of precise finger detection by using dual Kinects. Whereas the original Kinect was very much a full-body sensor intended for bodies up to twelve feet away from the camera, 3Gear have made the Kinect into a more intimate device. The user can pick up digital objects in 3D space, move them, rotate them, and even draw free hand with her finger. The accuracy is amazing. The founders, Robert Wang, Chris Twigg and Kenrick Kin, have just recently released a beta of their finger-precise gesture detection SDK for developers to try out and instructions on purchasing and assembling a rig to take advantage of their software. Here’s a video demonstrating their setup and the amazing things you will be able to do with it.

oblong

Mastering the technology is only half the story, however. Oblong Industries has for several years been designing the correct gestures to use in a post-touch world. This TED Talk by John Underkoffler, Oblong’s Chief Scientist, demonstrates their g-speak technology using gloves to enable precision gesturing. Lately they’ve taken off the gloves in order to accomplish similar interactions using Kinect and Xtion sensors. The difficulty, of course, is that gestural languages can have accents just as spoken languages do. Different people perform the same gesture in different ways. On top of this, interaction gestures should feel intuitive or, at least, be easy for users to discover and master. Oblong’s extensive experience with gestural interfaces has aided them greatly in overcoming these types of hurdles and identifying the sorts of gestures that work broadly.

brekel-kinect-pro-face

The advent of the Kinect is also having a large impact on independent film makers.  While increasingly powerful software has allowed indies to do things in post-production that, five years ago, were solely the provenance of companies like ILM, the Kinect is finally opening up the possibility of doing motion capture on the cheap.  Few have done more than Jasper Brekelmans to help make this possible.  His Kinect Pro Face software, currently sold for $99 USD, allows live streaming of Kinect face tracking data straight into 3D modeling sofrtware.  This data can then be mapped to 3D models to allow for realtime digital puppetry. 

Kinect Pro Face is just one approach to translating and storing the data streams coming out of the Kinect device.  Another approach is being spearheaded by my friend Joshua Blake at Infostrat.  His company’s PointStreamer software treats the video, depth and audio feeds like any other camera, compressing the data for subsequent playback.  PointStreamer’s preferred playback mode is through point clouds which project color data onto 3D space generated using the depth data.  These point cloud playbacks can then be rotated in space, scrubbed in time, and generally distorted in any way we like.  This alpha-stage technology demonstrates the possibility of one day recording everything in pseudo-3D.

Got an Image Enhancer that can Bitmap?

Every UI platform needs a killer concept.  For the keyboard and mouse it was the Excel sheet.  If you ever watch the rebooted Hawaii Five-0, you’ll realize that for Touch it’s the flick.  Flicking is more satisfying than tapping on sooo many levels.  Birds do it, bees do it, even monkeys in the trees do it.

Gestural interfaces haven’t found that killer concept yet, but it may just be the ability to zoom in on an image.  Like flicking and entering tabular data, killer concepts don’t necessarily have to be clever.  They just have to feel right.

Consider what John Anderton spent his time doing in 2002’s Minority Report.  For the most part, he used innovative fantasy technology (later made real at Oblong Industries) to enhance images on his rather large screen.

Go back even further and you’ll recall Rick Deckard used speech recognition to enhance an image in 1982’s Blade Runner.  This may be the first inkling any of us had of the true purpose of NUI.

It obviously left an impression on the zeitgeist because every movie or TV show attempting to demonstrate technological sophistication on the cheap (CSI being the biggest culprit) managed to insert an “enhance” scene into their franchise somewhere.

And if you happened to have a movie with no budget, there was no reason you should let this stop you.

And while we’re getting nonstalgic for NUI, let’s not forget to give credit where credit is due. Before Leap Motion, before Microsoft’s Kinect, before Oblong’s g-speak, even before Minority Report, there was the NES Power Glove:

And in the decades after, all we’ve managed to do is to enhance that killer concept.

What’s In Kinect for Windows SDK 1.5?

You shouldn't have come back, Flynn.

Microsoft has just published the next release of the Kinect SDK: http://www.microsoft.com/en-us/kinectforwindows/develop/developer-downloads.aspx  Be sure to install both the SDK and the Toolkit.

This release is backwards compatible with the 1.0 release of the SDK.  This is important, because it means that you will not have to recompile applications you have already written with the Kinect SDK 1.0.  They will continue to work as is.  Even better, you can install 1.5 right over 1.0 – the install we take care of everything and you don’t have to go through the messy process of tracking down and removing all the components of the previous install.

I do recommend upgrading your applications to 1.5 if you are able, however.  There are improvements to tracking as well as the depth and color data.

Additionally, several things developers asked for following the initial release have been added.  Near-mode, which allows the sensor to work as close as 40cm, now also supports skeleton tracking (previously it did not). 

Partial Skeleton Tracking is now also supported.  While full body tracking made sense for XBox games, it made less sense when people were sitting in front of their computer or even simply in a crowded room.  With the 1.5 SDK, applications can be configured to ignore everything below the waist and just track the top ten skeleton joints.  This is also known as seated skeleton tracking.

Kinect Studio has been added to the toolkit.  If you have been working with the Kinect on a regular basis, you have probably developed several workplace traumas never dreamed of by OSHA as you tested your applications by gesticulating wildly in the middle of your co-workers.  Kinect Studio allows you to record color, depth and skeleton data from an application and save it off.  Later, after making necessary tweaks to your app, you can simply play it back.  Best of all, the channel between your app and Kinect Studio is transparent.  You do not have to implement any special code in your application to get record and play-back to work.  They just do!  Currently Kinect Studio does not record voice – but we’ll see what happens in the future.

Besides partial skeleton tracking, skeleton tracking now also provides rotation information.  A big complaint with the initial SDK release was that there was no way to find out if a player/user is turning his head.  Now you can – along with lots of other tossing and turning: think Kinect Twister.

Those are things developers asked for.  In the SDK 1.5 release, however, we also get several things no one was expecting.  The Face Tracking Library (part of the toolkit) allows devs to track 87 distinct points on the face.  Additional data is provided indication the location of the eyes, the vertices of a square around a player’s face (I used to jump through hoops with OpenCV to do this), as well as face gesture scalars that tell you things like whether the lower lip is curved upwards or downwards (and consequently whether a player is smiling or frowning).  Unlike libraries such as OpenCV (in case you were wondering), the face tracking library is using rgb as well as depth and skeleton data to perform its analysis.

I fight for the Users!

The other cool thing we get this go-around is a sample application called Avateering that demonstrates how to use the Kinect SDK 1.5 to animate a 3D Model generated by tools like Maya or Blender.  The obvious way to use this, though, would be in common motion capture scenarios.  Jasper Brekelmans has taken this pretty far already with OpenNI and there have been several cool samples published on the web using the K4W SDK (you’ll notice that everyone reuses the same model and basic XNA code).  The 1.5 Toolkit sample takes this even further by, first, having smoother tracking and, second, by adding joint rotation to the mocap animation.  The code is complex and depends a lot on the way the model is generated.  It’s a great starting point, though, and is just crying out for someone to modify it in order to re-implement the Shape Game from v1.0 of the SDK.

The Kinect4Windows team has shown that it can be fast and furious as it continues to build on the momentum of the initial release.

There are some things I am still waiting for the community (rather than K4W) to build, however.  One is a common way to work with point clouds.  KinectFusion has already demonstrated the amazing things that can be done with point clouds and the Kinect.  It’s the sort of technical biz-wang that all our tomorrows will be constructed from.  Currently PCL has done some integration with certain versions of OpenNI (the versioning issues just kill me).  Here’s hoping PC will do something with the SDK soon.

The second major stumbling block is a good gesture library – ideally one built on computer learning.  GesturePak is a good start though I have my doubts about using a pose approach to gesture recognition as a general purpose solution.  It’s still worth checking out while we wait for a better solution, however. 

In my ideal world, a common gesture idiom for the Kinect and other devices would be the responsibility of some of our best UX designers in the agency world.  Maybe we could even call them a consortium!  Once the gestures are hammered out, they would be passed on to engineers who would use computer learning to create decision trees for recognizing these gestures much as the original skeleton tracking for Kinect was done.  Then we would put devices out in the world and they would stream data to people’s Google glasses and … but I’m getting ahead of myself.  Maybe all that will be ready when the Kinect 2.5 SDK is released.  In the meantime, I still have lots to chew on with this release.

Famous Youtubers: from our far-flung correspondent

Still not recovered from book writing, I have asked my eleven year old son to provide an overview of what’s going on in YouTube land.  My son spends a lot of time working on his own videos – mostly guides to Minecraft and short Lego stop-motion films – and looks up to the sort of people who have managed to eek out a living doing this.  Here are some of the movers-and-shakers in his world:

Hello audience, I am Paul Ashley; son of James Ashley… I am writing this article because of my epic writing skills I gained at school! Oh also because my dad said to… My Youtube account is PaulVAshley so remember to subscribe to me! Or don’t… Let’s begin our Top 5 Most subscribed Youtubers!

#5: Freddiew (Freddie Wong) 3,022,460(as of now) Subscribers.

Freddie and Brandon are two good friends who enjoy making videos with sweet VFX. I’ve always liked their videos, and I still do. I was first introduced to the channel by my friend ANONYMOUS. Umm… okay… anyway, he wanted to show me a tutorial Freddie and Brandon made on First Person Shooter Videos. I began watching all of his short movies starting with “Mr. Toots.” I have become one of his biggest fans. I also wonder what he has in store for us in “Video Game High School.” He is a great director and he is my role model!

#4: Machinima 4,356,027(as of now) Subscribers.

Machinima is an actual company that employs people to play games all day and occasionally make a “machinima” (A video with voices filmed from a game) from time to time. I think this channel is slightly unfair because they have hundreds of people making their videos. I enjoy certain songs that they make, but most videos I think to be just plain stupid. This is only my opinion though… Overall, I really like them only they sometimes have a video that is “bad.”

#3: Smosh (Ian Hecox and Anthony Padilla) 4,464,823(as of now) Subscribers.

Smosh is definitely my personal favorite Youtube channel. They upload new videos every week. Ian has a separate channel for making shows called “Ian is bored,” and “Lunchtime with Smosh.” I was first introduced by a few friends, one of them being Sam. Anyways, we would watch the “Theme song” series (Mortal Kombat, Pokémon, Teenage Mutant Ninja Turtles…). That was back in ’07 or ’08. Nowadays, they upload sketch videos. Overall, I love all of their videos except a few crappy ones.

#2: Nigahiga (Ryan Higa) 5,256,220(as of now) Subscribers.

Nigahiga… The most popular, classic Youtuber of all time! He is definitely the most famous among Youtubers. I was first introduced by my friends Shirish, Sam, and ANONYMOUS. We enjoyed videos like “How to be Ninja, Gangster, and Nerd.” My favorite video is “THE BEST CREW: The Audition.” I almost died in laughter. Overall, I like ALL of his videos.

#1: RayWilliamJohnson 5,408,244(as of now) Subscribers.

FINALLY! I’ve been enslaved to write this article for HOURS! So… where were we… Ah yes, lucky number 1. Ray is a Youtuber that makes a web show called =3. I was first introduced by my friend Shirish. Mr. Johnson (hehe) used to entertain me when I was 10, but I’ve grown ever-so bored of his predictable jokes. He also has a channel called “BreakingNYC.” Ahem, now this boy-man is funny to the creepy weirdoes of Youtube. Overall, I hate to sound sketchy but I dislike all of his videos.

YAY! ENDING PARAGRAPH! I like all of the channels I reviewed except RWJ. Okay, bye guys that’s all you get.

-From the insane mind of Paul Vladimir Ashley.

Concerning Old Books

There are few things sadder than a pile of old technical books. They live on dusty bookshelves and in torn cardboard boxes as testament to the many things we never accomplished in our lives. Some cover fads that came and went before we even had time to peruse their contents. Others cover supposedly essential topics we turned out to be able to program perfectly well without – topics like algebra, geometry and software methodology … [continued]

Kinect and the Atlanta Film Festival

Tomorrow, I will be appearing at the Atlanta Film Festival on a panel (link) moderated by  Elizabeth Strickler of the Georgia State Digital Arts Entertainment Lab.  The panel is called Post Production: How to Hack a Kinect to Make Your Own Motion Controlled Content and will be at the Landmark Midtown Art Cinema on March 28th at 2:45.  The other panelists are Ryan Kellogg, creative lead for Vivaki’s Emerging Experiences group, and Tara Walker, a Microsoft Kinect evangelist.

Minecraft 1.2.4: How to Change Your Skin

Like many fathers, after my son turned seven I regretfully no longer had any idea what he did from day to day.  To my surprise, I recently found out that my eleven year old son posts video tutorials to YouTube.  I’m extremely proud and just a little bit concerned.  Here is some of his work: