James Ashley

Magic Leap One vs HoloLens v1 Comparison

October 8, 2018October 15, 2018 James Ashley14 Comments

I’m currently sitting in my room at the L.A. Grand Hotel waiting for the L.E.A.P. conference to start. I’ve been holding off on this comparison post because I had promised Dennis Vroegop I would give it first as a talk at the Techorama Netherlands conference – which I did last week. I will do a feature comparison based on publicly available information, then highlight features unique to the Magic Leap, and then distinguish subtle but important differences that only become apparent from spending months with these devices at the developer level. Finally I want to point out design improvements in the Magic Leap that are so good for Mixed Reality that I predict they will be incorporated into the next version of HoloLens.

Keep in mind that this is a comparison of two different generations of devices. The Magic Leap One is coming out two years after the HoloLens and would be expected to be better. At the same time, the HoloLens v2 is being released some time in 2019 and can be expected to be better still.

1. Field of View

In raw numbers, the field of view of the Magic Leap One is approximately 25% better than the HoloLens. The HoloLens field of view is estimated to be about 29-30 degrees wide and 17 degrees high. The Magic Leap One is 40 degrees wide by 30 degrees high. There is a corresponding difference in resolution, with the HoloLens offering 1268 by 720 per eye and the Magic Leap One providing 1280 by 960 per eye.

The Magic Leap One uses the same wave guide display technology that the HoloLens does, however, so how did they pump up the FOV? First, the ML1 has a more powerful battery than the HoloLens does, and it’s often been claimed by Microsoft that FOV is largely dependent on the power of the projection. This is probably offset, though, by the fact that the ML1 is using more power to project in two planes instead of only one like the HoloLens does (with 6 Waveguide layers compared to 4 in the HoloLens).

Another trick is that the waveguides in the Magic Leap are closer to the wearer’s eyes than they are in the HoloLens. As a consequence, you can wear glasses underneath the HoloLens while you cannot do so comfortably under the Magic Leap device.

In addition to this, Jasper Brekelmans and Dennis Vroegop suggested over coffees along the Amstel River (in a conversation about David Copperfield) that because one’s peripheral vision is closed off in the ML1, the perceived FOV may be even larger than the actual. The theory behind this is that, due to the widespread use of glasses, we have become used to not paying attention to our peripheral vision so much and consequently are comfortable with this tunneling of our vision.

Blocking off the peripheral field of view might cause issues in certain industrial settings, but the general effect is that what you can see as a proportion of your overall FOV is much larger in the ML1 than it is in the HoloLens. Or another way of putting this is that the empty areas of your FOV, as a proportion of your available FOV, is much smaller than it is in the HoloLens.

On top of this, the aspect ratio of the FOV in the ML1 is much taller than in the HoloLens, which may end up doing a better job at accommodating vertical seccaddic movements of the eyes.

Because of the narrower gap between the device and the wearer’s eyes, the Magic Leap can’t accommodate glasses as the Hololens can. To compensate, Magic Leap is developing relationships with online eyeglass manufacturers to provide prescription inserts that can be placed in front of the waveguides and magnetically lock into place. There’s some controversy over whether this is a good or a bad thing. Some developers have expressed concern that this will make demoing Magic Leap at events more difficult than demoing HoloLens, since those with poor vision will either not be able to participate or, alternatively, we will be forced to carry around a large suitcase of prescription inserts to every event.

On the other hand, when I think of what MR will be like in the future, I tend to think of them resembling real glasses (and not electronic contacts, which simply scare me). When they reach the size and ubiquity of modern glasses, it will make sense for each person to have their own personalized device with their appropriate prescription. Magic Leap is on the right track in this case. It’s just in the intervening period that we have to figure out how to share our limited, expensive devices with others.

	HoloLens v1	Magic Leap One
Price	$3000 – $5000	$2300
OS	Windows	Android variant
Field of View	~30 deg x 17 deg	40 deg x 30 deg
Resolution	1268 x 720 per eye	1280 x 960 per eye
Depth Sensor	Time of Flight	Time of Flight
Display Type	Wave Guide	Wave Guide
Hand Gestures Recognized	2	9
Underlying comic book technology	Light Engines	Light Fields
Controller	Click	6 DOF
Hand tracking	limited	fingers (3 joints each)
Processing unit	Above nose	Light pack
Audio	Spatial Sound	Spatial Sound

2. Hardware Specs (It’s all about the battery)

HoloLens v1	Magic Leap One
Intel Atom x5-Z8100 1.04 GHz Intel Airmont (14nm) 4 Logical Processors 64-bit capable	NVIDIA® Tegra X2 SOC 2 Denver 2.0 64-bit cores + 4 ARM Cortex A57 64-bit cores (2 A57’s and 1 Denver accessible to applications)
8086h (Intel)	GPU. NVIDIA Pascal™, 256 CUDA cores; Graphic APIs: OpenGL 4.5, Vulkan, OpenGL ES 3.3+
2GB RAM	8GB RAM
64GB Storage	128GB Storage

The Magic Leap One is overall a much beefier machine than the current HoloLens. While both the HoloLens and the Magic Leap One advertise a 3 hour battery life, these can mean vastly different things. In order to drive all of its extra hardware, the Magic Leap One needs a much beefier battery. The ML1 is powered by a twin-cell battery with 36.77 Wh, running at 3.83 V. The HoloLens has a 1.65 Wh battery.

For overall performance, the larger battery means the world meshes (i.e. surface reconstruction, world mapping) are much denser and more frequently updated on the Magic Leap than on the HoloLens. The Time-of-Flight depth camera can fire off more frequently and for longer periods.

The larger battery and beefier specs also translate to much better 3D performance. The HoloLens is able to run 30,000 polygons at 60 fps. Beyond that, the fps begins to drop. The Magic Leap runs upwards of 1 million polygons at 60 fps.

On the downside, that more powerful battery rig needs a fan to cool it whereas the HoloLens is passively cooled. In laboratory and medical scenarios where a sterile environment must be maintained, active cooling with a fan could be a problem.

3. The HoloLens and Tracking

The HoloLens uses 4 monochrome cameras (“environment aware sensors”), an accelerometer, magnetometer and gyroscope in a sensor fusion configuration, and a custom HPU to perform head tracking. The Magic Leap one has a similar setup minus the HPU.

The HoloLens tracking is still somewhat better than the ML1’s. It loses tracking less frequently and digital content is less jittery when seen up close or while the wearer is in motion.

Overall, though, tracking performance is fairly close between the two devices.

4. Magic Leap Extras

The ML1 has a couple of features that are simply outside of the box. One is the eye tracking. There are inward facing cameras that track the wearer’s eye movements as invisible IR flashes.

The tracking is not continuous and is captured at a much lower resolution level than the displays. While they shouldn’t be used for direct user interactions, they are great for providing context for other interactions. It would be great if someone would write a keyboard that uses eye tracking to select keys. In the meantime, I wrote this heat vision demo that uses eye tracking to burn the walls of my house — I think of it as “Superman with a Migraine”. Note the eye-blink tracking.

The other cool extra in the Magic Leap is two planes of focus. Most VR devices have a single plane of focus at infinity. The HoloLens has a single plane of focus set at two meters.

In the magic leap one, when you look at near objects, objects further away (on the outer plane) seem to go out of focus. When you look at objects close up, the objects further away go out of focus. I would guess that the close plane is around a meter and the out one about 3 meters but I’m not really sure. In the Lumin OS .91, there is also a sporadic green shift in the near plane (which I expect will be fixed soon).

5. The Tether

The Magic Leap One is made up of two parts: the Light Pack and the Light Wear. They are connected by a cable. The Light Wear contains all the sensors, projectors and displays while the Light Wear, worn at the hip, contains all the computer bits and the battery.

This is an engineering choice that allows for a much larger power source. Without the tether solution, a large battery would not be possible. Without the large battery, the ML1’s enhanced depth sensing, improved graphics processing and larger field of view would not be possible.

In addition, this design makes the Magic Leap a much more comfortable fit on the head. The weight distribution is better than on the HoloLens, it is lighter, and it doesn’t require extra straps.

The tether solution is actually so effective that I would be surprised if the HoloLens v2 does not follow a similar design. The original one-piece “tetherless” solution Microsoft came up with for the HoloLens was visionary, but severely limiting.

6. Developing

If you have ever developed in Unity for the Android (or really any other device) then you know how to develop for Magic Leap in Unity. You press a button and your app compiles to an .mpk image (Android uses “.apk” file extensions). If your device is attached, you can deploy directly by clicking on “build and run”.

Magic Leap apps can also be built with the Unreal Engine.

HoloLens apps run on a Unity player sandboxed in a UWP app. The development cycle consequently involves exporting your HoloLens app as a Visual Studio project targeting UWP and then building and deploying in UWP. In general (and it may just be me) this has been tedious.

It became even worse when the immersive WinMR devices (or occluded WinMR – basically Microsoft VR) devices came out last year and the basic tools used for HoloLens development, known as the HoloLens Toolkit and then the Mixed Reality Toolkit, was expanded to supported both kinds of device. Because of some issues with Unity, building for WinMR required certain versions of Unity and above while developing for HoloLens required certain versions of Unity and below. And this state went on for several months to the point that finding the correct Windows SDK paired with the right MRTK version paired with the correct Unity version became a closely kept alchemical formula passed from developer to developer.

This experience may not be the same for everyone but it left me a bit traumatized. By contrast, Magic Leap development is simply a pleasure. I can build and see the results very quickly in my device. I can wear the device for hours at a time. I typically only stop development when the ML battery runs down and I have to let it recharge. I don’t have a Magic Leap Hub, which would allow me to charge while I dev, but I intend to get one.

The Magic Leap toolkit is still not quite as capable as the open source Mixed Reality Toolkit managed by Stephen Hodgson and others.

The Magic Leap also has a simulator rather than an emulator for developing without a device. This actually makes sense since the Hololens emulator runs the HoloLens OS in a virtual machine, which might be tricky given the much larger specs of the Magic Leap.

7. Interactions

The Magic Leap supports robust hand and gesture tracking as well as a 6DOF controller. The DOF in 6DOF stands for degrees of freedom. We know not only the direction the controller is pointing in (3DOF) but also its position.

I love the controller. I love it so much it made me finally admit to myself that I hate the HoloLens tap gesture. No one ever gets it right. It’s awkward. It’s uncomfortable and makes me feel like I’m performing a kung fu move.

By contrast, a controller just makes sense. The UX for MR, I believe, should always support three layers of interactions. Mixed reality UX should support hand gestures for ease of use. It should fall back to the controller for precision movements. It should finally fall back on the delta pad on the controller for accessibility.

For all of my antipathy toward the HoloLens tap, however, I have to say I miss the HoloLens bloom gesture (escape), which I keep trying to use in Magic Leap to no avail. Instead, in Magic Leap holding the controller’s Home button for three seconds is the escape gesture, which I don’t really like. It also bothers me that hand gestures aren’t supported in the core desktop (the Icon grid) – but this is still the Creator’s Edition (translation: dev edition) after all.

[Late edit thanks to SH: it should also be pointed out that the Lumin OS (the desktop layer) currently doesn’t support hand gestures, which I find baffling. For now, you can’t get past the login and other initial screens without a paired phone or a controller.]

Summing Up

So is the Magic Leap One better than the HoloLens v1? Oh yes. By leaps and bounds.

1. The development workflow is much more straight forward and pleasant.

2. The increased battery size and beefier hardware makes it possible to do things, performance wise, that the HoloLens tended to stop us from doing. Phone and tablet level experiences are doable now.

3. The Magic Leap One has a much better interaction model than the HoloLens does. How did anyone ever do MR without a controller? (Actually, everyone used an XBox controller in the end in order to get any sort of real work done, but we don’t talk about that much.)

Is it time to jump back into Mixed Reality development?

If you spent $3.2K to $5K for a HoloLens, then you owe it to yourself to spend $2,300 for a Magic Leap. It’s the device you originally wanted. The HoloLens was a brilliant device back in 2016 and really the first of its kind, but it had limitations. Many of the projects you were never able to realize in HoloLens (in the small dev community that developed around HoloLens, we all know what these are) are now doable with the improved Magic Leap specs. Additionally, your enterprise stories are much easier to sell with the controller. Instead of spending 5 minutes of your precious pitch time explaining how tap works, you can now just let your potential investors and clients go straight into the demo with a controller they basically already know how to use.

Is there a future in spatial computing?

Now there is. There was a brief pause between 2016 and the middle of 2018, but we currently have two great devices available with another shoe dropping soon. Microsoft will be coming out with a HoloLens v2 sometime in the first half of 2019 which I would predict will implement the tethered design Magic Leap is using. This will be an improvement over the current Magic Leap which in turn will be driven to improve its own tech.

Microsoft has an advantage because it started this journey back in the Kinect days and has the resources of Microsoft Research to draw on. Magic Leap has an advantage because, well, they aren’t Microsoft and don’t face the internal political problems a large tech giant does (though no doubt they have their own). More importantly, they have their own U.S.-based production lines (as well as production lines in Mexico) and are less reliant on China, which hopefully means they are capable of much quicker turn-arounds and initial SKU production.

When do we get smaller devices that wear like glasses?

I have no idea, but try to think in terms of 3, 5, 10 years. We always overestimate what can be done in 3 years but always underestimate how much things will change in 10. Somewhere in the middle, we will intersect with our MR futures.

Your comments, corrections and criticisms are welcome in the comments below. I’ll try to keep up with them and incorporate what you say into the main article as appropriate.

Thank you Atlanta Code Camp 2018

September 18, 2018December 13, 2018 James Ashley

I was invited to speak at Atlanta Code Camp on September 15th. I spoke about mixed reality and was fortunately able to run some demos. When I got out of standard business app dev a few years ago and began specializing in VR on MR, one of the unfortunate side-effects was that I saw a lot less of the community (Mixed Reality tends to be more of a global community, for whatever reason, that convenes online rather than in person). Coming in for this event is one of my great chances to meet up with old friends.

In the speakers room, we talked about military apocalypse preparedness, the difficulty of getting jobs on the dark net, the advantages of trio programming (a better version of pair programming), and being paid in bitcoin.

Mixed Reality As An Art Form

June 29, 2018June 29, 2018 James Ashley

First a homework assignment and then a request. Please watch a few of these http://kogonada.com/ video essays on cinema.

I’d like to put together something like this for AR and VR. The basic idea is to take something small in [AR | Cinema | Sci Fi television] and then develop the topic into a conversation. But I need your help with ideas.

For instance, consider the flick in Sci Fi movies but also in Hawaii Five-Oh and in the Avengers movie used to send a piece of digital media from one screen to another. Where did it come from? How did it develop as a gesture? Has it changed over time? Where is it used in real world apps? Does it make sense as good UX or is it pure Sci Fi stuff?

Alternatively, what is the the relationship between John Carpenter’s They Live (long overdue for a big budget remake), The Matrix and spatial computing?

Alternatively, what does Beaudrillard’s ‘desert of the real’ tell us about VR?

Alternatively, what sort of H-C interactions do we need before head-mounted-displays could begin to compete with smartphones as portable computers and begin to fulfill their financial promise.

So I need help with ideas to explore, in essay style, with suggestions for media that can be used to explore.

Once we have enough, and if there is enough interest, I’ll work with you on producing these essays as video essays and we’ll put together an online book about MR, with each of you as an essay author and providing narration.

It’s at minimum a six month project but something I feel the world needs and we are uniquely positioned to create. I’m a little tired of both book writing and recording technical videos when good design thinking around MR is what we’re most lacking in right now.

So, my dear readers, have you got any ideas that you have been keeping to yourselves about the design and philosophy behind MR but needed help expressing?

My VRLA Shopping List

May 10, 2018May 10, 2018 James Ashley1 Comment

I meant to finish this earlier in the week. I spent the past weekend in Los Angeles at the VRLA conference in order to hear Jasper Brekelmans speak about the state of the art in depth sensors and visual effects. One of the great things about VRLA is all the vendor booths you can visit that are directly related to VR and AR technology. Nary a data analytics platform pitch or dev ops consulting services shill in sight.

Walking around with Jasper, we started compiling a list of how we would spend our Bitcoin and Ethereum fortunes once they recover some of their value. What follows is my must-have shopping list if I had so much money I didn’t need anything:

1. Red Frog HoloLens mod

First off is this modified HoloLens by Red Frog Digital. The fabrication allows the HoloLens to balance much better on a user’s head. It also applies no pressure to the bridge of the nose, but instead distributes it across the user’s head. The nicest thing about it is that it always provides a perfect fit, and can be properly aligned with the user’s eyes in about 5 seconds. They designed this for their Zombie Maze location-based experience and are targeting it for large, permanent exhibits / rides.

2. Cleanbox … the future of wholesome fun

If you’ve ever spent a lot of time doing AR and VR demos at an event, you know there are three practical problems you have to work around:

seating devices properly on users’ heads
cleaning devices between use
recharging devices

Cleanbox Technology provides a solution for venue-based AR/VR device cleaning. Place your head-mounted display in the box, close the lid, and it instantly gets blasted with UV rays and air. I’d personally be happy just to have nice wooden boxes for all of my gear – I have a tendency to leave them lying on the floor or scattered across my computer desk – even without the UV lights.

3. VR Hamster Ball

The guy demoing this never seemed to let anyone try it, so I’m not sure if he was actually playing a hamster sim or not. I just know I want one as a 360 running-in-place controller … and as a private nap space, obviously.

4. Haptic Vest and Gauntlets

Bhaptics was demoing their TactSuit, which provides haptic feedback along the chest, back, arms and face. I’m going to need it to go with my giant hampster ball. They are currently selling dev units.

5. Birdly

A tilt table with an attached fan and a user control in the form of flapping wings is what you need for a really immersive VR experience. Fortunately, this is exactly what Birdly provides.

6. 5K Head-mounted Display

I got to try out the Vive Pro, which has an astounding 2K resolution. But I would rather put my unearned money down for a VRHero 5K VR headset with 170 degree FOV. They seem to be targeting industrial use cases rather than games, though, since their demo was of a truck simulation (you stood in the road as trucks zoomed by).

7. A globe display

Do I need a giant spherical display? No, I do not need it. But it would look really cool in my office as a conversation piece. It could also make a really great companion app for a VR or AR experience.

8. 360 Camera Rig with Red Epic Cameras

Five 6K Red Dragon Epic Cameras in a 360 video rig may seem like overkill, but with a starting price of around $250K, before tripod, lenses and a computer powerful enough to process your videos – this could make the killer raffle item at any hi-tech conference.

9. XSens Mocap Suit

According to Jasper, the XSens motion capture fullbody, lycra suit with realtime kinematics is one of the best available. I think I was quoted a price something like $7K(?) to $13K(?) Combined with my hamster ball, it would make me unstoppable in PvP competitive Minecraft.

10. AntVR AR Head-mounted display

AntVR will be launching a kickstarter campaign for their $500 augmented reality HMD in the next few weeks. I’d been reading about it for a while and was very excited to get a chance to try it out. It uses a Pepper’s ghost strategy for displaying AR, has decent tracking, works with Steam, and at $500 is very good for its price point.

11. Qualcomm Snapdragon 845

qualcomm

The new Qualcomm Snapdragon 845 Chip has built-in SLAM – meaning 6DOF inside-out tracking is now a trivial chip-based solution – unlike just two years ago when nobody outside of robotics had even heard of SLAM algorithms. This is a really big deal.

Lenovo is using this chip in its new (untethered) Mirage Solo VR device – which looks surprisingly like the Windows Occluded MR headset they built with Microsoft tracking tech. At the keynote, the Lenovo rep stumbled and said that they will support “at least” 6 degrees of freedom, which has now become an inside joke among VR and AR developers. It’s also spoiled me, because I am no longer satisfied with only 6DOF. I need 7DOF at least but what I really want is to take my DOF up to 11.

12. Kinect 4

This wasn’t actually at VRLA, and I’m not ultimately sure what it is (maybe a competitor for the Google computer vision kit?) but Kinect for Azure was announced at the /build conference in Seattle and should be coming out sometime in 2019. As a former Kinect MVP and a Kinect book author, this announcement mellows me out like a glass of Hennessy in a suddenly quiet dance club.

While I’m waiting for bitcoin to rebound, I’ll just leave this list up on Amazon for, like, in case anyone wants to fulfill it for me or something. In the off chance that that actually comes through, I can guarantee you a really awesome unboxing video.

Is Ethical AI Ethical?

April 22, 2018April 22, 2018 James Ashley2 Comments

Is ethical AI ethical? This is not meant to be a self-referential question as much as a question about semantics. Do we know what we mean when we talk about ethics? And, as a corollary, can we practice ethical AI if we aren’t sure what we mean when we talk about ethics. (Whether we speak correctly about artificial intelligence is a matter we can examine later.)

Is it possible that ethics is one of those concepts we all think we understand well, agree that it is important to understand in order to lead good lives, but don’t really have a clear grasp of.

If this is the case, how do we go about practicing ethical AI? What is the purpose of it? How do we judge whether our ethical standards regarding AI are sufficient or effective? What would effective AI ethics look like? And is the question of ethical AI one of those problems we need to develop AI in order to solve?

Toward an ethical AI, here are some passages I consider important:

“Everyone will readily agree that it is of the highest importance to know whether or not we are duped by morality.” — Emmanuel Levinas, Totality and Infinity

“What is happiness?

“To crush your enemies, to see them driven before you, and to hear the lamentations of their women!” – Conan the Barbarian

“Imagine that the natural sciences were to suffer the effects of a catastrophe. A series of environmental disasters are blamed by the general public on scientists. Widespread riots occur, laboratories are burnt down, physicists are lynched, books and instruments are destroyed. Finally a Know-Nothing political movement takes power and successfully abolishes science teaching in schools and universities, imprisoning and executing the remaining scientists. Later still there is a reaction against this destructive movement and enlightened people seek to revive science, although they have largely forgotten what it was….

“The hypothesis which I wish to advance is that in the actual world which we inhabit the language of morality is in the same state of grave disorder as the language of natural science in the imaginary world which I described. What we possess, if this view is true, are the fragments of a conceptual scheme, parts of which now lack those contexts from which their significance derived.” — Alasdair MacIntyre, After Virtue

“A great-souled person, because he holds few things in high honor, is not someone who takes small risks or is passionately devoted to taking risks, but he is someone who takes great risks, and when he does take a risk he is without regard for his life, on the ground that it is not on just any terms that life is worth living.” – Aristotle, Nicomachean Ethics

“In the name of God, the Merciful, the Compassionate.

“Someone asked the eminent shaykh Abu ‘Ali b. Sina (may God the Exalted have mercy on him) the meaning of the Sufi saying, He who knows the secret of destiny is an atheist. In reply he stated that this matter contains the utmost obscurity, and is one of those matters which may be set down only in enigmatic form and taught only in a hidden manner, on account of the corrupting effects its open declaration would have on the general public. The basic principle concerning it is found in a Tradition of the Prophet (God bless and safeguard him): Destiny is the secret of God; do not declare the secret of God. In another Tradition, when a man questioned the Prince of the Believers, ‘Ali (may God be pleased with him), he replied, Destiny is a deep sea; do not sail out on it. Being asked again he replied, It is a stony path; do not walk on it. Being asked once more he said, It is a hard ascent; do not undertake it.

“The shaykh said: Know that the secret of destiny is based upon certain premisses, such as 1) the world order, 2) the report that there is Reward and Punishment, and 3) the affirmation of the resurrection of souls.” — Avicenna, On the Secret of Destiny

“The greatest recent event—that “God is dead,” that the belief in the Christian God has ceased to be believable—is even now beginning to cast its first shadows over Europe. For the few, at least, whose eyes, whose suspicion in their eyes, is strong and sensitive enough for this spectacle, some sun seems to have set just now…” – F. Nietzsche, The Gay Science (1887)

HoloLens and MR Device Dev Advisory Mar-2018

March 28, 2018March 29, 2018 James Ashley1 Comment

I’m currently doing HoloLens development with VS 2017 v15.6.4, Unity 2017.3.1p3, MRTK 2017.1.2, and W10 17128 (Insider Build).

Unity 2017.3.1p3, a patch release, includes the 2017.3.1p1 fix for hologram instability:

(993880) – XR: Fixed stabilization plane not getting set correctly via the SetFocusPointForFrame() API, resulting in poor hologram stabilization and color separation on HoloLens.

There continues to be uncertainty about whether this fixes all the stabilization problems or not – though it’s definitely better than it has been over the past several months.

UnityWebRequest continues to always return true for the isNetworkError property. Use isError or isHttpError instead. The older WWW class probably shouldn’t be used anymore. There are reports that media downloads aren’t working with UnityWebRequest while other file types are.

So if you have something working now, and have work-arounds in place, you probably shouldn’t upgrade. I know of HoloLens developers who are still very happy working in the older Unity 5.6.3.

April is shaping up to be very interesting. According to Unity, they will be releasing Unity 2018.1.0 then. For UWP/HoloLens developers, this means the addition of the .NET Standard 2.0 API compatibility level. .NET Standard 2.0 can be thought of as the set of APIs commonly supported by both .NET Core 2.0 (what UWP uses) and .NET Framework 4.6.1 (used for Windows apps and in the IDE). By supporting this, Unity 2018.1.0 should provide us with the ability to write much more common MonoBehaviour script code that works in both the IDE and on the HoloLens without using precompiler directives.

Of course, this is only useful if the HoloLens actually supports .NET Core 2.0, which is why the announcement of the RS4 Technical Preview is such a big deal. This is the first major firmware update for the HoloLens since release, and brings with it all the changes to Windows platform since the Anniversary Update (build 10.0.14393) which was also known as RS1 and which supports .NET Core 1.0.

Redstone 4 (build 10.0.17133), also known as the Spring Creators Update, is supposed to drop for PCs in mid-April. Which coincidentally is also when Unity 2018.1.0 is supposed to drop. So it would not be out of the question to expect a version of RS4 for HoloLens to drop at around the same time.

What sets RS4 for HoloLens apart from RS4 for Windows? For one thing, on the HoloLens we will have access to a new feature called Research Mode, providing access to low level sensor data such as the ToF depth camera and potentially the 4 mono cameras and the microphones. This in turn can be used to try out new algorithms beyond what the HoloLens already currently uses for data analysis.

On the UI front, the MR Design Labs interface tools have finally been integrated into the dev branch of the Mixed Reality Toolkit. Fingers crossed that this will make its way into the main branch in April also.

Finally, Magic Leap’s mixed reality headset, dubbed the Magic Leap One, had its debut at GDC this month. They also opened their creator portal to all developers, with links to documentation, the Lumin SDK, a special version of Unity 2018 to develop ML apps and a simulator to test gesture and controller interactions.

In the interest of full disclosure, I’ve been developing for the Magic Leap for a while under NDAs and inside a locked room ensorcelled by eldritch spells. It’s a great device and finally creates some good competition for the HoloLens team at Microsoft.

The first reaction among people working with the HoloLens and occluded MR devices may be to be defensive and tribal. Please resist this instinct.

A second, well-funded device like the Magic Leap One means all that much more marketing dollars from both Microsoft and Magic Leap spent on raising the profile of Mixed Reality (or Spatial Computing, as ML is calling it). It means healthy competition between the two device makers that will encourage both companies to improve their tech in efforts to grow and hold large swaths of the AR market. It also means a new device to which most of your spatial development skills will easily transfer. In other words, this is a good thing my MR homies. Embrace it.

And from the development side, there are lots of things to like about Magic Leap. Lumin is Linux/Mono based, which means a higher level of compatibility between the platform and pre-existing Unity assets from the Asset store. It also supports development in Unreal. Lastly, it also supports development on a Mac, potentially offering a way for crossover between the design, gaming and enterprise dev worlds. This in turn raises interest in high-end AR and will make people take a second look at HoloLens and the occluded MR devices.

It doesn’t take a weatherman to know it’s going to be a great summer for Mixed Reality / Spatial Computing developers.

I’m leaving Facebook because blah blah ethics

March 21, 2018March 21, 2018 James Ashley

I’ve deleted my Facebook account (which apparently takes anywhere from 24 hrs to 45 days to accomplish) over their involvement with Cambridge Analytica. I’m not saying you should too, but you should think about it. For me it came out of distaste at colluding with a large company whose sole purpose is to collect personal data and monetize it. The data is being collected for current but primarily future AI applications – I think there’s little doubt about that. Also Facebook has known about its privacy and trust problems for a long time and has been warned internally about it but chosen to do nothing until they got caught.

At an ethical level (rather than just moral squeamishness on my part) I’ve been advocating for greater ethical scrutiny of matters touching on AI, so it would have been hypocritical to let this pass. That made it an easy decision on my part (though still painful) but is somewhat unique to my situation and past statements and doesn’t necessarily apply to any of you.

https://www.theatlantic.com/technology/archive/2018/03/facebook-cambridge-analytica/555866/

But at least it’s a good moment to pause, consider and exercise our ability to think ethically. What is the case for staying with Facebook? What is the case for leaving Facebook?

Here’s my first go at an ethical case:

For staying:

Facebook helps me connect with family and old friends in a convenient way. I might lose these relationships if I delete Facebook.

As a Microsoft MVP, I’m judged on my reach and influence. Deleting Facebook amputates my reach and consequently my appeal to Microsoft as an MVP.

Facebook is used to organize people against repressive regimes in liberation movements. This is a clear good.

For leaving:

Cambridge Analytica was not an isolated instance. Facebook was a poor steward of our personal information.

Facebook was warned about its policies and didn’t act until 2015 to review its internal practices.

This is Facebook’s core business and not an aberration. They are collecting data for AI and had a strong interest in seeing how it could be analyzed and used.

Participating in Facebook gives them access to my information as well as that of my friends.

Their policies can change at any time.

Participating in Facebook encourages family and friends to stay on Facebook.

The loose relationships formed on Facebook encourage and facilitate the spread of false and/or skewed information.

Facebook served as a medium for swaying the 2016 U.S. elections through bots and Russian active measures.

Analysis:

The harm outweighs the gains. The potential for future harm also outweighs the potential for future gains. Also, I am co-implicated in present and future harm promulgated by Facebook through continued participation. QED It is ethically necessary to sever relations with Facebook by deleting my account.

Meta-analysis:

But just because something is ethically wrong, does that make it wrong wrong?

Alasdair MccIntyre proposed in his 1981 book After Virtue that we post-moderns have lost the sense for moral language and that worldviews involving character, moral compasses and virtue are unintelligible to us, for better or worse.

Malcolm Gladwell makes a contiguous point in a talk at New Yorker Con about how we justify right and wrong to ourselves these days in terms of “harm” (and why it doesn’t work).

In a past life, I used to teach ethics to undergraduates and gave them the general message that everyone learns from Ethics 101 in college: ethics is stopping and thinking about not doing something bad just before you are about to do it anyways.

Along those lines, here are some arguments I’ve been hearing against acting on our ethical judgment against Facebook:

1. It will not really harm Facebook. If a few hundred thousand North Americans leave Facebook this week, it will be made up for by a few hundred thousand Indians signing up for Facebook next week.

2. Facebook is modifying its policies as of right now to fix these past errors.

3. Facebook has just severed relations with Cambridge Analytica, who are the real culprits anyways.

4. Since 2015, you’ve had the ability to go into your Facebook settings and change the default privacy permissions to better secure your private information.

Which are legitimate points. Here are some counter-points (but again this is just me doing me while you need to do you. I’m not judging your choices even if I sound judgy at the moment):

1. Leaving Facebook isn’t primarily about imposing practical consequences on Facebook (though if it did that would be gravy) but rather about imposing consequences upon myself. What sort of person am I if I do not leave Facebook knowing what they have done?

2. It isn’t even primarily about what bad deeds Facebook has committed in the past but rather about what those actions and policies say about who Facebook is today. Facebook is, to use a term the author Charles Stross coins in his talk Dude, You Broke the Future, a slow AI. It is a large corporation which has an internal culture that leads it to repeatedly follow the same algorithms. There is no consciousness guiding it, just algorithms. And the algorithms collect widgets, in economic terms – which in this case is your personal data. It can do nothing else. Eventually it wants to turn your personal data into cash because of the algorithm.

3. In the moral languages of the past, we would call this its character. Facebook has a bad character (or even Facebook is a bad character). Having good character means having principles you do not break and lines you do not cross.

4. I want to wear a white hat. To that purpose, even if I can’t stop a bad character, I don’t have to help it or be complicit in its work.

Switching gears, somewhere between World War I and World War II, we probably lost the proper language to discuss ethics, values, morals, what have you. There are artifacts from that time and before, however, that we can dig up in order to understand how we used to think. Here’s one of my favorites, from the e. e. cummings poem i sing of Olaf glad and big:

Olaf(upon what were once knees)
does almost ceaselessly repeat
“there is some shit I will not eat”

This is what I would say to Mark Zuckerberg, if he would listen, and even if he won’t.

The AI Ethics Challenge

March 14, 2018March 29, 2018 James Ashley2 Comments

A few years ago, CNNs were understood by only a handful of PhDs. Today, companies like Facebook, Google and Microsoft are snapping up AI majors from universities around the world and putting them toward efforts to consumerize AI for the masses. At the moment, tools like Microsoft’s Cognitive Services, Google Cloud Vision and WinML are placing this power in the hands of line-of-business software developers.

But with great power comes great responsibility. While being a developer even a few years ago really meant being a puzzle-solver who knew their way around a compiler (and occasionally did some documentation), today with our new-found powers it requires that we also be ethicists (who occasionally do documentation). We must think through the purpose of our software and the potential misuses of it the way, once upon a time, we anticipated ways to test our software. In a better, future world we would have ethics harnesses for our software, methodologies for ethics-driven-development, continuous automated ethics integration and so on.

Yet we don’t live in a perfect world and we rarely think about ethics in AI beyond the specter of a robot revolution. In truth, the Singularity and the Skynet takeover (or the Cylon takeover) are straw robots that distract us from real problems. They are raised, dismissed as Sci Fi fantasies, and we go on believing that AI is there to help us order pizzas and write faster Excel macros. Where’s the harm in that?

So lets start a conversation about AI and ethics; and beyond that, ML and ethics, Mixed Reality and ethics, software consulting and ethics. Because through a historical idiosyncrasy it has fallen primarily on frontline software developers to start this dialog and we should not shirk the responsibility. It is what we owe to future generations.

I propose to do this in two steps:

1. I will challenge other technical bloggers to address ethical issues in their field. This will provide a groundwork for talking about ethics in technology, which as a rule we do not normally do on our blogs. They, in turn, will tag five additional bloggers, and so on.

2. For one week, I will add “and ethicist” to my LinkedIn profile description and challenge each of the people I tag to do the same. I understand that not everyone will be able to do this but it will serve to draw attention to the fact that “thinking ethically” today is not to be separated from our identity as “coders”, “developers” or even “hackers”. Ethics going forward is inherent in what we do.

Here are the first seven names in this ethics challenge:

I want to thank Rick Barraza and Joe Darko, in particular, for forcing me to think through the question of AI and ethics at the recent MVP Summit in Redmond. These are great times to be a software developer and these are also dark times to be a software developer. But many of us believe we have a role in making the world a better place and this starts with conversation, collegiality and a certain amount of audacity.

Windows Insider Builds and Unity Licenses

February 20, 2018February 20, 2018 James Ashley1 Comment

On one of my dev PCs, I’m on the Windows 10 Insider Builds inner ring, which updates my computer with intermediate builds of Windows. This gives me access to the latest features for testing. Unfortunately, it also means that my Unity Plus license goes into an inconsistent state every few weeks after an Insider Build changes my computer profile enough that Unity no longer recognizes it as the same computer.

In the past I’ve tied going into my account management and try to revoke the assigned seat and then assign it again to myself. Sometimes this works but often it doesn’t. Somehow during the Windows update process, something happens with the original license that it cannot be disabled correctly.

I recently found out that there is a different route to deactivating seats that will work even after a Windows 10 update. Back in the account landing page, you need to navigate to My Seats instead of going through the Organizations menu.

This leads you to a page that lets you remove all license activations. Once you’ve clicked on “Remove all my activations”, you can then successfully use your license key to reactivate your Unity IDE.

Reminder to pre-buy your Black Panther tickets

February 3, 2018 James Ashley

Tickets are actually already selling out for the new Black Panther movie, in part because of buy outs of complete theaters for children like these Ron Clark Academy students in Atlanta:

For those who don’t know (bad nerds) the Black Panther is a superhero in the Marvel Universe who is also the king of the African nation of Wakanda. Wakanda is secretly the most technologically advanced country in the world, the sole source of vibranium in the world (magic metal, Captain America’s shield is made of it), but projects an image of just being another African nation in order to avoid interference. In the marvel universe it had an all out war with the Submariner’s Atlantean army a few years ago and currently auteur Ta-Nehisi Coates is taking a turn at writing the series and problematizing it (which I don’t totally like but tastes and all that).

Deadly_Hands_of_Kung_Fu_29

The history of the series is basically the usual Marvel thing – Marvel takes advantage of racial trends and exploits them (like with Luke Cage, Iron Fist and Shang-chi) and end up creating something kind of miraculous. In this case, a kingdom of black people who are more advanced than anybody else, culturally and technologically.

wonder-woman-4

If I can talk race and gender a little (feel free to squirm) according to a friend, it does for black people what she assumes Wonder Women did for white women. You get to see yourself in an ideal way without any cultural or political baggage. How do you create a movie hero without any cultural baggage or identity politics attached – you create a fictional country like Themyscira or Wakanda and make your characters come from there – that way they don’t become walking political arguments but instead just _are_.

thorteamup

So after seeing Wonder Woman, my wife asked me if that’s what it’s like for men to see movies, and I think, yeah, pretty much. I’m not Thor, but he’s an ideal projection of myself when I watch the movie and he gets to drink and carouse and hang out with his buddies and women admire him and no one ever neggs him for it. And my wife said she’s learned to watch and enjoy those kinds of movies but Wonder Woman showed her what that experience could really be like.

fury

At the risk of overselling — Black Panther is going to do that for race, according to a friend who got to go to the Hollywood premiere. No white guilt, no resentment, no countries getting called sh* holes, just gorgeous, powerful black people and a reprieve from our crazy mixed up world for a while. Plus, again according to the friend, it’s also another fun Marvel movie.

b_mirror

And here’s the catch for lovers of VR and AR – obviously there’s going to be lots of great Cinema4D faux-holograms used to show how advanced Wakanda is. Not only did Marvel movies pioneer this, but holograms are the chief way movies and tv show “advanced” societies (e.g. Black Mirror, Electric Dreams).

But more importantly, when we talk about “virtual” immersive experiences I think we implicitly know it means more than just having objects in a 3D space. The world is a given and stuck thing, while virtual reality frees us from that and lets us see it differently. The killer AR/VR app is going to do that at a very deep level. I think Black Panther is going to provide an ideal/target/goal for what we want to achieve with all of our headgear. An artificial experience that alters the way we see reality – if only for a few hours plus the afterglow period. Great virtual reality needs to alter our real reality – and make it better.