James Ashley

Changes in Kinect SDK Beta 2

November 5, 2011 James Ashley

To celebrate the one year anniversary of the Kinect, Microsoft has launched a new Kinect website and released the Beta 2 version of the Kinect for Windows SDK: http://www.kinectforwindows.org/ .

This is not the commercial license we have been waiting for (it is reported to be coming in early 2012) but truly the next best thing. The Beta 2 SDK introduces many performance improvements over the Beta 1 that was released in June.

With the improvements also come some alterations to the basic syntax for instantiating the core objects of both the Nui and Audio namespaces, though these changes also have side-effects that will likely affect your code. In particular, I want to cover one substantial change in the Nui namespace and one substantial change in the Audio namespace.

In the Beta 1, it was standard to instantiate a Nui.Runtime object in order to configure applications to read the depth, color and skeleton streams. In a WPF application, the code looked like this:

        Microsoft.Research.Kinect.Nui.Runtime _nui;

        public MainWindow()
        {
            InitializeComponent();

            this.Unloaded += (s,e) => _nui.Uninitialize();
            _nui = new Runtime();

            _nui.Initialize(RuntimeOptions.UseColor 
                | RuntimeOptions.UseDepth);
            _nui.VideoFrameReady += _nui_VideoFrameReady;
            _nui.DepthFrameReady += _nui_DepthFrameReady;

            _nui.VideoStream.Open(ImageStreamType.Video, 2
                , ImageResolution.Resolution640x480
                , ImageType.Color);
            _nui.DepthStream.Open(ImageStreamType.Depth, 2
                , ImageResolution.Resolution320x240
                , ImageType.Depth);

        }

In the Beta 2, the Nui.Runtime has been obsoleted. Instead, the Runtime type provides a static collection called Kinects that returns a Runtime object for each Kinect connected to the PC. Additionally, in the new Beta, Runtime objects cannot be configured and used in the Initialize method as was possible with the Beta 1. Instead, this must be done in the Loaded event handler. This has ended up breaking a lot of my Beta 1 code. Fortunately the refactor is fairly easy. The following code assumes there is only one Kinect sensor plugged into the PC.

        Microsoft.Research.Kinect.Nui.Runtime _nui;

        public MainWindow()
        {
            InitializeComponent();
            this.Loaded += (s, e) =>
                {
                _nui = Runtime.Kinects[0];
                _nui.Initialize(RuntimeOptions.UseColor);
                _nui.VideoFrameReady += _nui_VideoFrameReady;
                _nui.VideoStream.Open(ImageStreamType.Video
                , 2
                , ImageResolution.Resolution1280x1024
                , ImageType.Color);
                };
        }

A central difficulty in working with the audio namespace is that the DMO named the KinectAudioSource, which is as fundamental to audio processing for the Kinect as the Nui.Runtime is for video and skeletal processing, must be instantiated on a thread running in a multithreaded apartment model. WPF applications, unfortunately, run in an STA thread. This required a bit of additional wiring up in the Beta 1 to create the KinectAudioSource object on a separate thread.

With the Kinect for Windows SDK Beta 2, this is implicitly taken care of for us. If you have already been working with the Beta 1 audio namespace, you will finally be able to take out the workarounds you created to program against the audio stream along with all the additional care required for dealing with a multithreaded application.

[Thanks to Clint Rutkas of Coding4Fun for correcting me on the difference between obsolescence and breaking changes. The new Runtime instantiation process is not technically a breaking change.]

Book Announcement

September 20, 2011 James Ashley

My colleague Jarrett Webb and I have reached the halfway point on our book for APress this week, so it finally seems safe to announce that we are working on Beginning Kinect Programming with the Kinect SDK to be released towards the end of the year. My original plan for September and October was to blog feverishly on the Kinect SDK, but given the tyranny of the book schedule, I’m finding it hard not to put everything related to the Kinect into the book instead. Towards the end of the process, however, we’ll likely have odds and ends that do not make sense in a Beginner book that will, nevertheless, want to be said. Those odds and ends will eventually make their way here.

In the meantime, the big news is WinRT. Besides the fact that everyone and their neighbor is already writing their reflections on BUILD, obviating any need for me to contribute, I’ve actually already written a BUILD summary for Razorfish, which should appear on www.emergingexperiences.com shortly.

I can’t resist commenting, all the same, that while I have javascript and C# under my belt, C++ is in my near future. No love or time for F#, sadly.

The Build up to BUILD

August 30, 2011 James Ashley5 Comments

I recently found out in a Louis Menand New Yorker review of The Yale Book of Quotations edited by Fred Shapiro (republished in The Best American Essays of 2008 edited by Adam Gopnik, another New Yorker contributor – was the fix in? – which I happened to come across while browsing at a going-out-of-business sale in the local Borders) that the phrase “Shit happens” first appeared in print in 1983 in a publication called UNC-CH Slang. For some reason, I thought the phrase antedated the year the Apple IIe was launched.

An argument can be made that the start of the build up to the BUILD conference is intimately tied to the rise of Apple, the on-again off-again relationship between Steve Jobs and Apple, the strange periods when Microsoft helped keep Apple afloat in order to maintain a legitimate competitor in the face of government investigations, and of course the release of a largish iPod Touch device known as the iPad which inaugurated the post-PC period.

Alternatively, we can say that the build up began during Bob Muglia’s interview with Mary Jo Foley at PDC 2010 (to which BUILD is the successor conference) and his fateful comment concerning Silverlight:

“… our strategy has shifted …”

The quite vociferous Silverlight community loudly denounced Bob Muglia, questioning his authority to declaim on the matter (he was in fact the President of the Microsoft Server and Tools Division at the time), as well as Mary Jo Foley for reporting it. The genie was only put back in the bottle when John Papa, a Microsoft Silverlight evangelist, quickly threw together the Silverlight 4 Firestarter to explain the Silverlight roadmap and Pete Brown, another Silverlight and WPF evangelist, started a more even-keeled and civil dialog about the matter on his blog.

Approximately a year later we find ourselves in the same position. As Bob Muglia indicated in last year’s interview, Silverlight continues to be an important platform for Windows Phone development and will have niches in video streaming and line-of-business applications. Silverlight 5 is still slated for release sometime in 2011. Overall, however, announcements from the Windows Team indicate that a greater emphasis will be placed on HTML5, JQuery and CSS 3 as tools for developing on the new Windows 8 OS while the role of .NET going forward has been more muted. All eyes are on the upcoming BUILD conference on September 13th to find out not only what the Windows 8 \ Tablet strategy from Microsoft will be, but tangentially what the future of Silverlight and .NET in general will be.

(A question can profitably be raised here concerning why the uproar from the Silverlight community was so loud when the WPF community, arguably much larger, effectively rolled over at rumors and signs that the vision for WPF was being curtailed and amid proclamations that WPF was dead. Certainly part of the reason was that WPF developers could generally also see themselves as Silverlight developers and at the time, due to the superior branding of the Silverlight platform, many were in fact trying to do that on their resumes. But where does one go from Silverlight? As interesting as Silverlight for Windows Phone is, it still doesn’t carry the technical gravitas that being a Silverlight developer did. Jupiter/DirectUI might be that next technology, but it would have been reassuring had Steven Sinofsky and his team said a bit more about it. At the same time, many developers who stuck with WPF looked upon the Silverlight developers with a distinct lack of pity and a certain amount of schadenfreude as their world appeared to be crumbling around them.)

Instead of revealing the developer story on the official Windows 8 blog, Microsoft has mostly been mum about the contents of the BUILD conference. The official agenda has been left blank since it was first posted, while unofficial information about Windows 8 has mostly come from sites like WinRumors and Mary Jo Foley’s articles on ZDNet , information like Windows 8 is being built for ARM chips allowing it to be deployed on a greater variety of devices (though apparently not on phones since Windows Phone continues to be developed on its own branch), that it may not support CD Rom drives (really?), that it has a new optional Metro-themed shell for navigation, and that it supports native development in HTML as well as some future XAML-based platform called either Jupiter or DirectUI.

The official blog, in the meantime, has mostly dribbled out pieces on an Office-like interface for Windows Explorer and a new look for copy/paste. They are apparently taking a page out of Steve Jobs’ book and attempting to create mounting excitement by saving all the good stuff up for one big reveal. One can envision a reception similar to the original introduction of the iPhone, though skeptics might also call to mind the mayhem and setting of theater seats on fire that accompanied the premier of Jean Renoir’s The Rules of the Game (privately I hope for this sort of response to an announcement so innovative that it simultaneously creates both fawning fans as well as fulminating foes seething with rage). In either case, the fact that Microsoft is taking this theatrical approach seems to indicate that they have something they believe is going to turn heads.

The challenge and expectations for Windows 8 resembles a sea of giant crabs: prickly and coming from all sides. The new operating system needs to present a broad new vision. It must at the same time be familiar enough that it doesn’t alienate either our grandmothers or the heads of our IT departments. It must work on a tablet (oh yes, it must!) as well as a desktop, while at the same time it must not sacrifice performance. It must sport a new consumer facing look and live up to expanded consumer expectations while at the same time support the slew of legacy applications we have individually and corporately invested in.

In my small corner of the development world, it should also attract the broad range of web developers and designers that have for years avoided anything having to do with Microsoft while also placating the many Microsoft-based developers that have been so loyal to Microsoft over the past decade (though, admittedly, they had their reasons and did quite well for themselves by playing in Microsoft’s stack).

This last point is often overlooked in the debates about how unfortunate Windows 8 might be for Silverlight developers. Attracting the sets of skills and non-Microsoft thinking available in the traditionally non-Microsoft community would be the most revolutionary thing Microsoft has done in a long time. In general, these are people who not only know how to work well with designers but, in smaller shops, are expected to be developer-designer hybrids. It’s a way of looking at the world that the Microsoft community greatly needs in order to reinvigorate the way we do development as we are increasingly expected to build not only secure and stable applications, but also experiences that look as good as the ones people are seeing on their consumer devices – even as the clean separation we’ve always understood to exist between UIs for games and UIs for apps continues to evaporate. Bringing these developers into the Microsoft fold would be a huge treasure drop for Microsoft and certainly worth upsetting a few developers to accomplish. It is also oh … so … difficult, as we found out with the Expression tools over the past few years.

So what will the new Windows 8 be? Here are three possibilities going from the least revolutionary to the most, all of which will undoubtedly be wrong.

1. A New Shell: Son of Gadget

There is an official video available showing off a new look for Windows that ZDNet reports is called the MoSH or Modern Shell that people can opt out of if they prefer a “classic” look. It may also be called “Mosaic” – another code name floating around. It is Metro based and supports multi-touch, resembling the Metro look of Windows Phone in many ways including the tile based design. This was actually an idea publicly proposed around the time Windows Phone was first revealed. Why not simply build a Metro shell on top of Windows 7 and release it as a tablet solution?

Underneath the shell will be a typical next generation Windows supporting Silverlight, WPF, a ribbon bar, and all the legacy technologies and apps we have grown comfortable with. This would be a distinctly old wine into new skins strategy. This is easily doable and Microsoft has experience with it from the “gadgets” concept introduced with Vista and extended in Windows 7. Like gadgets, MoSH components can be developed in a variety of technologies – which would explain some of the discussion around HTML5 vs Jupiter as rival development platforms for Win 8.

2. Web Based Apps: Everything 365

Microsoft has been building the technology around web-based solutions with their Office 365 cloud services, allowing users to employ either web-based versions of office or native apps. Extending this more generally to consumers would allow light weight software on tablets while making full versions of the apps available to desktop users. It would also explain the emphasis on HTML5 as lots of legacy apps would have to be ported to the web in order to make them work on lower-end hardware. On the other hand, we know that even sophisticated applications like Photoshop, for instance, can work over the web.

Why not everything else?

3. Virtual OS: Legacy Mode

This is my fantasy scenario. With Windows 7, Microsoft introduced something called XP Mode, which would allow users to run legacy apps on a virtual machine behind the scenes but have it look like you were running apps in the host OS. Could this concept be extended? One of the things we currently know officially about Windows 8 is it has hyper-V 3 integration of some sort. What if the Jupiter/HTML5 piece really is the biggest part of the new OS (a truly new game changing and light-weight OS) while XP Mode type functionality will be used to support legacy apps transparently for our grandmothers and IT managers.

This would be a remarkable sleight-of-hand, reminding me of a Penn & Teller routine where they go through several layers of misdirection to create the illusion of a man standing by a light pole smoking a cigarette when neither the cigarette or the light pole or the man are real.

What if this could be combined with the Microsoft cloud computing strategy to host VHD’s in the cloud so a person could access their cloud-based OS from any device – a truly Web OS? VMWare, after all, just recently introduced in a similar concept for something they are calling Project AppBlast. It wouldn’t be a huge stretch to believe that Microsoft is thinking along the same lines – especially if it provides an automagical solution for the backwards compatibility conundrum.

That would certainly be something to see as well as something that solves the problem of having a modern consumer look while preserving the power and functionality of traditional and legacy applications. In a scenario like this, all of our current development skills would still be relevant, from Silverlight and WPF to Visual Basic 6 and FoxPro. They would only be second class citizens on the OS to the extent that they are now virtual, rather than native, skills.

But I’m sure the reality will manage to be both more as well as less exciting than any of these suggestions … and the big reveal is only a few weeks away at this point. If I had to give a summary judgment of whether Microsoft’s wait-and-see strategy around Windows 8 has worked, I would have to say yes, absolutely.

As frustrating as the wait has been, the feints, misdirections and long silences have only wetted my appetite to find out what Windows 8 is hiding under the hood.

remix, thanks and glass beads

August 15, 2011 James Ashley

On the 6th of August, the Atlanta community put on a conference called ReMIX South based loosely on the aspirations of the MIX conference held each year in Las Vegas to highlight new technology, web design and other stuff.

Here are some quick statistics – we had roughly 438 attendees at ReMIX this year. We also had 23 speakers and lots of sponsors onsite.

ReMIX garnered an estimated 2000 tweets – estimated because our archive programs capped out at 1500. ReMIX apparently managed to trend nationally on August 6th on Twitter.

We tried our best to make the conference primarily about the attendees, the speakers and the sponsors.

Now that the event is over, however, I would like to take a moment to thank the people who put the conference together in roughly six weeks.

The main organizers were Cliff Jacobson, Sean Gerety, Dennis Estanislao, Farhan Rabbi, and Wells Caughey.

In addition, we had great logistical help from Corey Schuman, Linda Gerety, Ted Jacobson and Glen Gordon as the event unfolded and in the post-event period.

Additional help was provided by Joel Johnson, Ginny Caughey, Jonathan Marbutt and Shawn Wildermuth during the phone garage and also by Erin Gerety, Sophia Ashley, Paul Ashley, and Sharesh Vadali who put the registration packets together the night before ReMIX.

I want to highlight some of the skills and labor the organizers put towards the event. They all did more than I can possibly recall, but certain things stand out.

Cliff Jacobson took care of our finances, as well as all the negotiating with the hotel, logistical planning with the hotel and A/V. It was a lot and we basically just piled more responsibilities on him as the event approached since he never seemed to say no. In short, the event never would have happened without his organizational skills. In the post-event period, Cliff has also taken on the editing of our video footage and – as far as we can tell – has only had 3 hours of sleep a night for several weeks.

Sean Gerety used his persuasive powers to bring together an amazing UX track, organize the speaker dinner, and smooth over the bumps. Sean is one of the best connected people in the Microsoft UX community and we used his connections for all they were worth.

Dennis Estanislao designed and maintained our website, which was the anchor for the event and, until the day of the event itself, perhaps the only proof that ReMIX South was actually happening. As many people have learned in the past, a good website is the backbone of a good conference and keeping it up to date and accurate is a fulltime job.

Farhan Rabbi was essential to making this a true cross-discipline and bi-partisan event. Throwing a conference that is attractive to both microsoft as well as non-microsoft people, attractive to both CRUX and developers, is no easy task. Putting the lie to Kipling’s statement that never the twain shall meet, Farhan made sure that ReMIX was both ecumenical and catholic while at the same time gathering all the local talent for the amazing HTML5 track.

Wells Caughey threw himself into doing whatever was needed to get us over the finish line. In addition to pulling together the mobile track (admittedly the most difficult of the tracks to plan and find speakers for) he also managed our commons area.

Linda Gerety handled the registration for us, staying up late the night before to prepare and manning the tables the whole day. Corey Schuman manned the registration tables, took photos of the event, and is helping to get the recordings we managed to capture of the talks up for streaming. Both Linda and Corey did airport runs for us to pick up speakers. Also thanks to Zach Pousman for the after party at Eclipse di Luna and the celebration of the world wide web’s 20th anniversary. Thanks also to Kristina McInerny who just started helping us out spontaneously as well as Dave Ward and Ben Von Handorf for building phone apps for reMIX.

The thanks they receive for all this work is pretty much just some glass beads and a pat on the back. If you see them around and happen to have enjoyed the remix experience, please let them know.

Glass beads also play a peripheral role in the organization of ReMIX. The Glass Bead Game is the title of Nobel Laureate Hermann Hesse’s 1943 book about an intellectual game of the future based on montage and pastiche – basically putting dissimilar things side-by-side and seeing what connections players can make between them.

This is what we attempted to do at ReMIX. We combined top local and national speakers presenting on a variety of technologies from both the Microsoft and non-Microsoft worlds, covering UX, design and development. It was curated, of course – we tried to be careful to put things together we thought would work well, but still … We put all these different communities together, crossed our fingers, and waited to see what would happen.

Our thanks go out to the attendees, the speakers and the sponsors for making our version of the Glass Bead Game a great success.

We’ll be updating the www.remixsouth.com website with links to slide decks, photos and video recordings over the next weeks. Check in frequently to get the latest changes.

Why ReMIX?

July 20, 2011 James Ashley2 Comments

On August 6th, the Atlanta developer community will be hosting ReMIX South, a conference for designers and developers. This is the second year the conference has been held in Atlanta. Early Bird tickets can be purchased at http://remixsouth.eventbrite.com . The official website is at www.remixsouth.com . Tickets are only $30 through June 28th.

There are lots of great conferences throughout the year such as MIX, An Event Apart and TechEd. These all tend to be extremely expensive, however. At the other end of the spectrum are community events such as MadExpo, CodeStock, the various code camps and DevLink. These are great, inexpensive grassroots level events. Anyone can speak and the agendas tend to be more or less random.

ReMIX is an attempt to create something in between these two extremes. We created an event that has the level of speakers you would typically see at all the former two to three thousand dollar events but at the price of a community event.

We do this by spending much of our time throughout the year at all of these other conferences trying to recruit speakers for ReMIX South. We spend half the year discussing who is a top speaker, who is a rising speaker, and what topics have become important in 2011. In other words, we spend the majority of our effort simply planning out our speakers the way a painter mixes colors or a chef blends flavors.

We do this in order to provide what is, to our minds, a unique and satisfying experience for our attendees. Of all the speakers we reached out to, only two of our must haves couldn’t make it: Bill Buxton and Robby Ingebretsen – both had prior engagements.

We keep prices low through very generous sponsorship as well as being very frugal with your money – though controversial, we don’t provide lunch or t-shirts. While these are standard for most conferences, we found that we can cut the price in half simply by leaving them out. We also keep your bottom line low by choosing a central location, The Marriott at Perimeter Center, which has free parking.

(As an aside, when I was at An Event Apart, I paid the same amount for my parking as the price of one early bird ticket to ReMIX: $30.)

The other thing we try to do at ReMIX is to provide a designer event that is friendly toward developers, as well as a developer event that is friendly toward designers.

It is also an event that we try to make welcoming for both Microsoft stack as well as non-Microsoft stack developers. We understand that, depending on where you come from, our agenda will always seem to lean too much in one direction or the other. To our thinking, this is a good thing. We want to bring the different communities together.

Non-Microsoft developers will get a bit of exposure to a world they tend to stay away from, while Microsoft stack developers will have their minds expanded to a world they are not familiar with. At the end of the day, we all leave knowing a little more about our craft than we did when we came and have a broader understanding of what our craft entails. Everyone moves out of their comfort zone and becomes stronger for it.

And if you don’t want to have your mind expanded, that’s cool, too. We have enough sessions to keep anyone inside their comfort zone, if that’s what they want.

Here is what we are offering this year:

The Keynote

Albert Shum is one of the most fascinating people currently working at Microsoft. He is part of the revolution within Microsoft that transformed their mobile strategy and placed, for once, design at the center of a new technology offering. Albert led the design team that created the much discussed “Metro” language used originally on Windows Phone and now, according to well-placed rumors, on the Surface 2 and Windows 8. If you are still confused about what “Metro” actually is, this is your best opportunity to find out – he’ll be at the conference all day and is very approachable.

The Web Track

This is the track we are perhaps proudest of. If you are a Microsoft stack developer, then you might think of HTML5 as a zombie-like infestation that is taking over and displacing all the technologies you are used to working with.

On the other hand, if you aren’t part of the Microsoft world, you probably are perplexed when people make a big deal about HTML5 and wonder if they are talking about CSS3 + JQuery.

So what we are offering in the Web track at ReMIX is a bunch of non-Microsoft stack web developers explaining HTML5 to both Microsoft and non-Microsoft developers. Brilliant, right?

J. Cornelius of CoffeeCup Software will start by presenting “HTML5: Yes Really.” The title is a joke and if you don’t get it, then you really need to attend. He will provide the opening overview of HTML5.

John Agan, who builds amazing web experiences for Epic Labs, will teach us about JQuery.

Josh Netherton of MailChimp will school us on CSS3 in “More Than Just Rounded Corners.”

Finally, we’ve invited August de los Reyes of Artefact to speak. If you aren’t familiar with August, he happens to have given the most impressive talk at MIX11 this year – and it was only ten minutes long!You can find a video of his MIX presentation here . At ReMIX South, August will be presenting an expanded version of his talk 21st Century Design.

Mobile \ Tablet Track

The past year has been spent pursuing the code-once dream for mobile development using tools like Mono, PhoneGap, the Adaptive Web and, most recently, HTML5. If you’ve been following the trends in iPhone, Android and Windows Phone, you’ll know that this has been a rocky and occasionally treacherous path. Not only do the different tools not always work … ahem … perfectly, but the rise of tablets is also making it clear that designing and developing for non-desktop computers is a lot more complex than just working with different form-factors. We’ve invited several Microsoft as well as non-Microsoft stack people to walk us through the variegated world of mobile and tablet development.

Douglas Knudson, Technical Architect for Universal Mind and organizer of the Atlanta Flex User Group, will show us how to use Adobe Air to target multiple mobile and tablet platforms.

Luke Hamilton, Creative Director at Razorfish, will speak on “The Interface Revolution” and cover how to work with all the new devices we are being confronted with as technology keeps progressing.

Shawn Wildermuth, a well known trainer and expert in Windows Phone development, will walk us through the new features being introduced in Windows Phone Mango.

Jeremy Likness will talk about his experience working with Silverlight for tablets. He will also discuss what we currently know about Windows 8, which is being promoted as a tablet platform that uses both HTML5 as well as a XAML-based language for development.

Rob Cameron, who was also with us last year, is a Microsoft Architect Evangelist. He will talk, among other things, about developing games for Windows Phone using XNA.

Windows Phone Garage

No pretense of non-Microsoft material here. Starting in mid- to late-August, the Windows Phone Marketplace will start accepting Mango apps. This full day dev garage will get you ready for that. Just bring a laptop and we’ll take care of the rest – by the end of the day you will have an application ready to start making you money.

Unlike other phone garages, this one will be surrounded by top talent in development and design as well as several Windows Phone development MVPs. If you would like their help, we’ll setup a sign-up sheet so you can arrange to get one-on-one advice about your app.

UX Track

The UX Track has always made the ReMIX conference stand apart from other conference. This is really the place where we invite speakers to talk broadly about a variety of topics which we place, loosely, under the UX rubric.

Let me point out, first of all, that all of our speakers are amazing. These are our rock stars. Rick Barraza has become an institution helping developers understand UX and design as well as trying to help devs and designers to work together. He spoke at MIX this year. Jenn Downs is simply cool and MailChimp, her company, is widely lauded for breaking new ground in connecting with customers by being hip, playful, cheeky and, of course, extremely useful. MailChimp has pretty much been invited to speak at every major conference this year. Zach Pousman and James Chittenden were both extremely popular speakers at last year’s ReMIX. Zach is an expert in both academic and practical UX, while James is a UX Architect for Microsoft Consulting – you probably didn’t even know there was such a thing. We are very lucky to have them back. Designers think Matthias Shapiro is a designer while developers assume he is a developer since he has been so effective in bridging both worlds. His talk on Motion is a must see.

If you have spent your careers as developers and have never been exposed to the world of UX and design, then the best favor I can do for you is to recommend that you spend your whole Saturday in this track. You’ll thank me for it. Really, you will.

Kinect Track

This is very exciting for us. The Kinect Track is our opportunity to take a new technology, bring together some of the leading experts on developing for the Kinect and hold the first conference event about the Kinect. Other conferences are beginning to have one or two Kinect talks a piece, if they have any. At ReMIX, we provide a full day of Kinect content.

All of our Kinect speakers come for the most part from the pages of the KinectHacks website.

Jarrett Webb is the creator of KinectShop, an application that has given us the best picture so far of how the Kinect and related technologies will one day be used in retail. He is generously providing the introductory talk on developing for the Kinect.

Zahoor Zafrulla is a Phd candidate at Georgia Tech. He is making breakthroughs in using the Kinect sensor for education. His particular interest is in using the Kinect to teach American Sign Language.

Steve Dawson and Alex Nichols wrote the DaVinci Kinect application in November of 2010 – shortly after drivers for building Kinect applications for the PC became available. It was one of the first apps recognized for successfully pulling off ‘The Minority Report’ effect and Microsoft later asked them to present it at the E3 conference to demonstrate what Kinect hacking is all about.

Josh Blake is the best known figure in the Kinect world. Besides being widely recognized as an authority on Natural User Interface concepts, he is also the founder of the OpenKinect community. There are few people who know more about the growth and future potential of the Kinect technology than Josh.

The final Kinect session of the day will be a panel discussion moderated by Josh Blake with panelists Albert Shum, Rick Barraza, Luke Hamilton and Zahoor Zafrulla. They will be discussing the influence of TV shows and movies on how we envision the future of technology as well as what the future of technology will actually look like.

Nerd Psychology

July 13, 2011 James Ashley1 Comment

Charles_Garnier_architect

At the beginning of this month I was invited to give a keynote at the MadExpo conference opposite Jeff Prosise, who gave an amazing Thursday keynote address. This was heady stuff and more due to the subject matter I proposed, “What Recent Breakthroughs in Nerd Psychology Can Teach Us About Software Development”, than my abilities as a public speaker.

The keynote attempted to address several intractable problems in software development using current psychological and neurological research:

Why hasn’t the proliferation of software frameworks and tools made software development go any faster?
Why does the obsolete technology we abandoned five years ago always eventually reappear as something new and trendy?
Why do developers find it impossible to predict how long a task will take until it is completed?
How much should a developer be paid for a mythical-man-month of work?
What is this fear of “coupling” that all software architects seem to exhibit?

In the process, I also tried to solve another set of interesting problems:

Whether the brain can be hacked
The secret of the Mona Lisa’s smile
Why multi-tasking is a myth
Why nerds like puns
Why the Turing Test is a red herring

The crux of the talk revolved around what is coming to be known as the Autistic Spectrum of Disorders, which includes: Autism, Asperger Syndrome and PDD-NOS (a catch all for disorders that do not meet the requirements for Asperger but approaches it).

Various research has connected these disorders with what is commonly known as “the nerd”. The two papers I cited in the talk were Ioan James’s Autism and Mathematical Talent and Nicholas Putnam’s Revenge or tragedy: Do nerds suffer from a mild pervasive developmental disorder? which was published in Adolescent psychiatry: Developmental and clinical studies, Vol. 17.

Asperger Syndrome and PDD-NOS are interesting here for the qualities these disorders share with what we would think of as a high functioning software developer. Though a bit laudatory, here is a quote from the pediatrician Hans Asperger on the disorder that bears his name:

It seems that for success in science or art a dash of autism is essential. For success the necessary ingredient may be an ability to turn away from the everyday world, from the simple practical, an ability to rethink a subject with originality so as to create in new untrodden ways …

Research on a neurological structure called mirror neurons, which were discovered in the late 80’s, are now believed to be associated with language acquisition and social aptitude. Some researchers identify a flaw in the functioning of mirror neurons with disorders like autism. If this is true, then it provides the clue as to why some people will turn away from social interactions and, in turn, have a large reservoir of concentration and obsessiveness which are turned to other, more mechanical, pursuits.

Pulling this talk together was deeply fascinating and I am indebted to my wife Tamara, a therapist as well as a highly accomplished hypno-therapist, for her help.

I also drew on several popular books for additional material. They are all very readable and fascinating:

The Tell-Tale Brain – V. S. Ramachandran

Sleights of Mind: What the Neuroscience of Magic Reveals About Our Everyday Perceptions — Stephen L. Macknik, Susana Martinez-Conde, Sandra Blakeslee

NurtureShock: New Thinking about Children – Po Bronson, Ashley Merryman

The Invisible Gorilla — Christopher Chabris, Daniel Simons

How We Decide – Johah Lehrer

The Language Instinct – Steven Pinker

… and while not directly used in this talk, a clear inspiration for myself and the other authors in this brief bibliography is the book which inaugurated the “everything you know is wrong” genre of non-fiction books:

Freakonomics – Steven D. Levitt, Stephen J. Dubner

The slides for this talk (which includes the Nerd-Geek-Dork Venn Diagram by Matthew Mason) can be found on my skydrive.

[For reference: a quick indicator of whether you are a nerd or a geek is this: if you can pull of horn-rimmed glasses, you are a geek. if you cannot pull off sunglasses, you are a nerd. I personally have trouble with sunglasses but, for some reason, keep trying.]

Windows 8 and the Two Worlds of HTML

June 17, 2011 James Ashley5 Comments

It’s a commonplace in literature that different people see the world in remarkably different ways. North Koreans and South Koreans, big-enders and little-enders, Morlocks and Eloi.

What often gets in the way of realizing this basic characteristic of the world is another — possibly modern and very democratic — tendency to believe that we can understand where anyone is coming from if we just try hard enough. There’s a brotherhood of man and all that, so if we simply sit down and discuss our differences, we should be able to all get along. Why can’t we all just get along?

One of the hardest things for Microsoft developers to understand is why everyone not-Microsoft hates Windows and the other MS products. We naturally assume these members of the web and design community are being irrational and remarkably biased. If they would just sit down at a Microsoft conference / sales pitch and get themselves educated, then everything would be better and we’d be one step closer to world peace.

This past week I had the opportunity – mostly by accident — to attend An Event Apart when it came through Atlanta – a conference for web designers and developers. What fascinated me was that all the issues and debates that I thought were important were not important at this conference.

First, I thought the most important thing going on with the web was HTML5. While HTML5 got some play at An Event Apart, it was basically treated as merely a tool. The real issue everyone was grappling with was the mobile web – whether it requires forking off of regular websites, or if it can be incorporated into the overall design and development process. Should there be one web for two windows, or a fractured web to support all the different devices coming to market.

Second, no one asked if Silverlight was dead. For all the angst expressed over the rise and decline of Silverlight in the Microsoft world, it was clear that, for the people at this conference, Silverlight had never made any inroads. And why should it – Silverlight was able to deliver content to Macs but never actually had design or development tools that could run on the Mac. The Silverlight story basically ignored all the designers and developers who had made Flash dominant for so long on the web.

So what did people want to know about? Whether Flash was dead. The answer, apparently, is no. Flash and HTML5 complement each other and … blah blah vampire emergency blah … you can fill in the blanks from all the things people typically say about Silverlight not being dead or Newt Gingrich’s presidential run not being dead.

Is IE the best HTML5 compliant browser? At An Event Apart I saw slide after slide claiming that cool web technique XYZ will not work on IE 7, 8 or 9. CSS animations, responsive web design, etc. all work great on Chrome, FF5 and Safari. Just not on IE. I had my laptop open for most of the conference and after a little time I found myself repeatedly using Firefox or Safari rather than IE to do my browsing – I just didn’t want anyone to know that IE 9 was my default browser.

And Windows 8? It never came up. People outside of the Microsoft worldview don’t understand the Win 8 story at all. For that matter, people inside of the Microsoft worldview also don’t understand it. All we do know is that HTML5 and Javascript will apparently be the preferred development platform for it.

Here’s the rub. Where are all these HTML5 and Javascript developers going to come from? Just as importantly, where are all the HTML5 designers going to come from – assuming that we want Win 8 apps to actually look good?

The HTML5 + jquery goodness story seems to be based on the notion that there is a huge reservoir of web developers out in the world that Microsoft can win over to becoming developers for their new operating system. Developers, developers, developers, right?

How is Steve Sinofsky and his PR team going to accomplish this? I’ve been through the looking glass; I’ve seen the other side and, quite frankly, those developers Microsoft wants to woo don’t want to have anything to do with Microsoft. While there’s an element of irrational contempt in all of this, it is also the result of many years of Microsoft not understanding the concerns of the other web community and even outright ignoring them.

Microsoft is possibly playing the longest game of hard-to-get anyone has ever seen. If that strategy doesn’t work, though, how is MS going to sex up Windows 8 for the other, non-Microsoft, web community?

Because without that other world of HTML, Windows 8 is going to fail.

Daydreaming about Kinect Fun Labs

June 12, 2011 James Ashley

cruise-minority-report

While at the CodeStock conference in Knoxville, I had the pleasure of talking with several indie XNA developers about the potential of the Kinect. They were discouraged, though, at the lack of opportunities for indie developers to create games for the Kinect. All I knew how to do was create XNA games and put them on a PC with a Kinect hooked up to it – which wasn’t quite what they were looking for.

Then last week at E3 Microsoft announced Kinect Fun Labs. Currently Fun Labs is a free download for the XBOX with what are basically a few concept apps. It includes Kinect Me, which scans the player and turns her image into a 3D model (though not, as has been reported, an XBOX avatar that can be used in the rest of the system). The Kinect then tracks the player and manipulates the model to follow the player’s movements.

Kinect Googly Eyes does basically the same thing with any rectangular object.

And then there are a few additional one-offs that can be downloaded from within the Kinect Fun Labs app.

What is interesting in all this are hints on the official website that we can upload our own apps at some point. This may be rumor only, of course, and it probably won’t fully satisfy indie developers who would like to make some money off of their games, but once we get past all the ifs and buts …

It would be very cool to have a way to share our Kinect apps – watching demos on Youtube only get us so far. If the XBOX also supports Silverlight at some point, we will have additionally have a development platform on which to quickly develop Kinect apps. It’s the stuff dreams are made of.

Telerik ToDoLists reference application

May 9, 2011 James Ashley20 Comments

Telerik has published a ToDoLists reference application on their site to demonstrate how to use their Windows Phone Rad controls as well as to walk developers through their process for designing a WP7 application. Telerik asked me to write a few words about it.

If you have already been through Microsoft’s default templates, this reference app is a good next stop. You can read about it and download the sample code here: http://www.telerik.com/products/windows-phone/getting-started/todolists.aspx

For all we learn from blog posts and books, there’s nothing quite like walking through a sample app to learn how to do something. There are several non-standard but – I think – extremely interesting things about this app that developers will find useful.

1. Telerik discusses their design process and includes a wireframe document for download. Many of us have no idea about how to go about designing a Windows Phone application. This little bit of guidance on the design process (definitely UI design first) is probably one of the most valuable things available to WP7 developers currently on the internet.

2. The ToDoLists app does not try to inherit the user defined accent colors or background theme. There is one look and it is the one the designer came up with. This may seem unorthodox at best and heretical at worst. To me, however, it seems like a brilliant move. The whole point of the limited customization of themes is to give the end-user a sense of control and personal connection with their apps. The cost of trying to incorporate the user selected theme, however, is a compromise of the design vision. If the design is good, I think it is worthwhile to simply ignore the user theme and maintain the integrity of the design as is done in this reference app.

3. The ToDoLists app also demonstrates how to organize the design vision in code and how to organize resources. This is a topic that usually gets short-shrift.

4. The ToDoLists app uses my friend Jeremy Likness’s Sterling NoSql database. This is extremely handy code to be able to walk through. The app even provides an example of generating initial data for Sterling.

5. It doesn’t use any of the standard MVVM frameworks. Instead, it’s a roll-your-own implementation of MVVM. This is a nice reminder that MVVM doesn’t have to be complex. We’ve almost reached the point when we no longer have to explain why MVVM should be used in XAML-based applications. It’s simply what we do, right?

6. This is an attractive design. It is recognizably Metro without slavishly following what most developers interpret as the Metro guidelines – that is, black, white and blocky.

So is this a paid ad for Telerik? Sort of – except to turn things on their heads, you get paid instead of me. If you are one of the first ten commenters on this post, and include a valid email, and are not a bot (so many conditions), Telerik will send you a free license to their RadControls for Windows Phone.

Just be one of the first ten commenters to say something like “I’d like a free license” in the comments by EOD next Monday (the 16th) and Telerik will hook you up.

The WPF Renaissance

April 18, 2011 James Ashley

The ‘Mysterious Stranger’ is a stock device in literature. Poe’s main character in The Masque of the Red Death is one instance of the theme. Mark Twain left us an unfinished novel built around another. An unexpected guest is central to one of the earliest works in English literature, Sir Gawain and the Green Knight. In Plato’s late dialogues, Socrates is replaced by the more enigmatic Xenon or Stranger. The old testament is populated by mysterious guests who turn out to be angels. The Venetian Carnival always potentially allowed commoners to mix with royalty since everyone was in disguise. Middle Eastern literature is filled with Tufaylîs or ‘unexpected guests’ who would provide additional entertainment to any party should they be allowed over the threshold. Medieval troubadours appear to have filled the same function in the West.

At the recent MIX 2011 keynotes in Las Vegas, amidst announcements about HTML5, IE9, ASP.NET MVC 3.x, Silverlight 5, Windows Phone Mango and the Kinect SDK, WPF made such an appearance. It was never called out, as such, but as each Kinect demo was presented on stage, WPF aficionados were beaming because they recognized their old friend behind the Kinect mask.

Why WPF? Because WPF is the natural workhorse for NUI development. From Kinect hacks to multi-touch displays to Surface technology, it is again and again the technology of choice providing both native integration with low level touch messages on the host OS as well as a powerful programming idiom. For many of us, it was the technology we learned to do MVVM on. We get to use the full resources of the .NET Framework with it as well as the ease of development provided by Visual Studio and Blend.

And if we are at the beginning of a NUI renaissance, as my friend Corey Schuman suggests in his MIX presentation, it follows that we are also in the midst of a WPF Renaissance as, for now, NUI and WPF are closely bound together.

To add sauce to the goose, over the weekend Rob Relyea from the WPF team tweeted the following elliptical comment:

“Yesterday wpf team had champagne/sparkling cider at our all hands meeting. Reached significant milestone.”

As pleased as I was with the Kinect demos at MIX, this piece of messaging took me completely by surprise. There are still WPF milestones? The Microsoft Tufaylîs are indeed gifted at entertaining and delighting.