“Pictures Under Glass” is Revolutionary, Not Transitional

I paused a few times yesterday while reading Bret Victor’s post on the future of interaction design. While I respect his intent to inspire us to dream up new interaction methods, I think he did readers a disservice in being so dismissive of “Pictures Under Glass” interaction.

To better understand Bret’s perspective in writing this post, I think it’s important to read an earlier post about his work at Apple: http://worrydream.com/Apple/

I’ll wait…

Here’s what stood out to me in that post: “I made many, many things. The ones I cared deeply about mostly didn’t catch on.” Apple is focused on the next 5-10 years of marketable consumer technology, but dreams about and experiments with technology that’s even further out. Since Bret is bound by NDA we may never know what he worked on at Apple, and it may very well have been revolutionary, but the fact that it didn’t catch on at Apple means we likely wont see anything like it succeed in the market near term. At least not at the scale of the iPhone or iPad. As Horace Dediu pointed out, the iPhone has undoubtedly revolutionized the mobile phone industry, and there’s a good argument to made that its Multi-Touch user interface was the driving force.

Here’s what I think is the fundamental problem with Bret’s argument: as humans we depend heavily on both visual and tactile feedback to interact with the physical world, but touch only “does the driving” once sight has helped develop patterns and coordination. Learning to tie a shoelace is incredibly difficult for a person who has no visual feedback, and even for someone who learned with the aid of sight, it takes years for the pattern to be so ingrained that it can be done without looking. I have no idea at what age most kids would be able to tie their shoes without looking, but I bet it’s 7 or 8, maybe older. If you have a younger child, give it a try and let me know.

The thing is, we already have more tactile computer input devices and we’ve been experimenting with them for decades: mechanical keyboards, midi enabled musical instruments, the mouse, joysticks, game controllers, Wii Remotes, the Power Glove, etc. And they all require time and skill to master. My 2 year old son, Luke, is incredibly adept at using the iPad precisely because it doesn’t require tactile feedback to master, not in-spite of it.

There are certain cases in which more tactile feedback may be able to enhance human-computer interaction without adding to the learning curve — such as the pneumatic displays Bret links to in his post — but I can’t get over the hunch that tactile interaction, like Siri and other voice input technologies, will augment and enhance Pictures Under Glass interaction, not revolutionize human-computer interaction.