My rambling about spatial computing prompted by Apple‘s Vision pro

I’ve been tentatively interested in Spatial Computing since University and I think they’ve been an interesting next step for user interface for a while now. Apple’s announcement of Vision Pro prompted some thoughts from me.

Computer interfaces and dimensions

If we look back you early computing interfaces you can see the teletypes as a one dimensional user interface, a 1D UI. You have this string of characters which you can in turn reply with your own words. At first they where printed on a typewriter like printer, but eventually they moved to screens. The basic idea of the command prompt was the same though.

Eventually we started to see graphical user interfaces. You know, windows, icons and so on. Pioneered by Engelbart, Xerox PARC and others and later brought to the masses with the original Macintosh. This 2D UI was the natural progression from the previous single dimensional UI.

The next logical step is 3D UIs. I think a lot of us have had a inkling of that. Remember the 3D file system browser in SGI IRIX featured in Jurassic Park?

In a sense you could say that the field of industrial design have dealing with 3D UIs for a long time. However, what set’s computing UIs apart is that they’re dynamic. The graphical user interface on say an iPad is a dynamic 2D UI.

I think the next step is a dynamic 3D UI.

Apple’s Vision Pro

For all it’s talk of Spatial Computing, Apple’s Vision Pro had little in the way of actual spatial computing. It’s virtual spatial environment was mostly taken up of flat planes. In other words: 2D UIs in a 3D world.

This might be an Apple Watch type situation. It was quite rough in the beginning, but eventually it did start to provide value. It’s different though in that it was a far more conservative UI than the spatial computing Apple touts with Vision Pro.

Another big problem with VR systems like the Vision Pro is that it’s isolating. I don’t see Apple meaningfully solving that in the short term with this specific product. A lot of people called it dystopian and compared it to Black Mirror for a reason. In the long term the realityOS might be for a play beyond these headsets as Steve Throughton-Smith mentions.

This is why for now I find research projects into 3D UIs more compelling.

Research into 3D UIs

There’s been academic research on this for a long time. One prominent example is MIT Media Lab’s Tangible Media Group. The idea here is to have dynamic media that works by dynamic physical 3D volume.

Another example is Dynamicland. This is a concrete project that aims to move computing out of our screens (in VR or otherwise) and into the physical world. It’s one of the more beautiful glimpses of the future I know of in the computing world. What’s especially wonderful about Dynamicland is it’s emphasis not only on the physical world, but on real life social computing. It’s the opposite of isolating.

Ultimately I find these versions of spatial computing to be far more appealing than Apple’s vision.

Back to Vision Pro

Does this mean that AR/VR systems like the Vision Pro don’t have it’s place? No. I do think for specific 3D oriented applications like architecture or industrial design these things can be useful. I know architects who love existing VR/AR gaming headsets for architecture usage. Both for letting clients previewing work, but additionally for doing design work (think sketching in 3 dimensions). There’s probably more abstract usages, like visualising mathematical concepts. Imagine learning about complex numbers using spatial computing. That’s a concept that’s pretty hard to represent merely in 2 dimensions.

However, my hope (and long timeframe expectation) is that the future of spatial computing exists in the physical world, not the screen based one. The world where your fellow people physically exists.