May 2014 – The Imaginative Universal

To build 3D movies with Kinect, you first have to import all of your depth data into a point cloud. A point cloud is basically what it sounds like: a cloud of points in 3D space. Because the Kinect v2 has roughly 3 times the depth data provided by the Kinect v1, the cloud density is much richer using it.

The next step in building up a 3D movie is to color in the pixels of the point cloud. Kinect v1 used an SD camera for color images. For many people, this resolution was too low, so they came up with various ways to sync the data from an DSLR camera with the depth data. This required precision alignment to make sure the color images lined up with and then scaled to the depth pixels. This alignment also tended to be done in post-production rather than in realtime. One of the most impressive tools created for this purpose is called the RGBD Toolkit, which was used to make the movie Clouds by James George and Jonathan Minard. The images in this post, however, come from an application I wrote over Memorial Day weekend.

Unlike its predecessor, Kinect for Windows v2 is equipped with an HD video camera. The Kinect for Windows v2 SDK also has facilities to map this color data to the depth positions in realtime, allowing me to record in 3D and view that recording at the same time. I can even rotate and scale the 3D video live.

trinity-matrix-opening

You’ll also notice some distortion in these images. I actually ran this 3D video capture on a standard laptop computer. One of the nicest features of the Kinect v2 is that it takes advantage of the GPU for calculations. If I don’t like the quality of the images I’m getting, I can always switch to a more powerful machine.

The next step, of course, is to use multiple Kinects to record 3D video. While I can rotate the current images, there are shadows and distortions which become more evident when the image is rotated to orientations not covered by a single camera. Two cameras, on the other hand, might allow me to do a live “bullet time” effect.

I don’t really know what this would be used for – for now it’s just a toy I’m fiddling with–, but I think it would at least be an interesting way to tape my daughter’s next high school musical. On the farther end of the spectrum, it might be an amazing way to do a video chat or to take the corporate video presentation to the next level.

europe knapsack

In my early twenties I travelled through Europe after graduating from college. The economy in the U.S. was not very good at the time and it seemed like a good thing to do before applying to grad schools. I learned quickly that it was a good idea to sew a small Canadian flag on my knapsack as it headed off potentially unpleasant experiences – for instance, getting overbilled for a meal. Apparently people assumed that Canada was not a particularly wealthy country and consequently Canadians didn’t face these issues.

The funny thing about the Canadians I ran into – besides the fact that they spoke exactly the way I did – was that once they sewed Canadian flags onto their knapsacks, they were surprised that they started feeling patriotic. They found this odd since a part of the Canadian identity at the time was that you just didn’t do the patriotism thing. That’s what countries further south did.

In turn, we Americans in Europe started to find ourselves not only missing mom and apple pie but also feeling these peculiar stirrings of the “P” word. So at that point how did we feel about our allegiance to our false flags?

In fact it didn’t much matter since the fun thing about traveling in a foreign country is that it never feels completely real. You feel as if you are always a spy spying on the native population and trying to act like you fit in when ifact you don’t really care. Vladimir Nabokov, the eternal exile, had a facile turn of phrase for this: “spies from Terra.” The only time I felt bad about this deep down sense of lightness and playfulness was when a Czech friend accused all Americans of being like children when it came to love. I didn’t know how to reply at the time. It only occurred to me later that this was true, but only when we were abroad flying false colors on our knapsacks.

I discovered later that when one returns to one’s home country, it is never fully your home country again. To some extent, you continue to be a spy from Terra with a knapsack and flying false colors. The knapsack is your soul and it is important to fill it with good things or you will lose yourself.

My metaphysical knapsack is invisible, of course, and I fill it up with books. The books remind me of who I am and who I want to be. Some I’ve carried around from over twenty years and some are newer.

The twenty year plus books include Doris Lessing’s Re: Colonised Planet 5, Shikaata which I found on a college bookshelf in Santa Fe, New Mexico. It also includes Robert Graves’s The White Goddess: A Historical Grammar of Poetic Myth. I’m not sure where I found that. Nabokov’s Ada is of course also there.

My metaphysical knapsack includes books I’ve never read but that I enjoy just thinking about and imagining the contents of like Merleau-Ponty’s The Visible and the Invisible as well as Raymond Queneau’s Exercises In Style. Brady Bowman’s Hegel and the Metaphysics of Absolute Negativity is also there. It is a brilliant book but I don’t actually understand it. It reminds me of a wonderful night I spent with my friend in Berlin in the 90’s. There’s also a trashy book in my knapsack called Shadowrun: Never Deal with Dragons by Robert Charrette – I read it in a seacoast bed-and-breakfast in Northern Wales and it captures a mood.

Everyone should have a metaphysical knapsack. It reminds you of who you are, who you were, and who you want to be. It’s a terrible thing to lose track of who you once wanted to be.

What’s in your knapsack?

Month: May 2014

3D Movies with Kinect for Windows v2

Metaphysical Knapsacks