Reminds me of Worldbuilder [1], a scifi short about a VR interface for editing the world around you. I kinda like the "grab object & kiss to save" gesture: it wouldn't make sense in 2D but feels natural in VR, although "grab & push object in your brain" might be easier implement to represent a save function. [this video]
The interesting part is that a lot of what he is talking about is already possible. - You can use augmented reality or VR with something like Oculus to display things in 3D space. - You can use Motion Capture technologies, plus maybe some convenient remote-control/keyboard-like devices to put a human in that virtual space and manipulate things. Only 2 technologies(Oculus + Mocap) would be enough to accomplish something like this: [this video] Give a bunch of people oculus rifts and put them in the same virtual reality - and we have the dynamic stage he is talking about. They don't even have to be in the same room of course, we have 3D realtime MMORPGs and second-life that already solved that problem!! You could attend a virtual talk by sitting in your apartment with oculus rift on your head. And of course "downloading a room" instead of a book is already very doable. We already have computer games and engines. That room is like a video game level. Downloading it is not a problem, to explore it all you need is oculus and a simple game controller or a keyboard/mouse. And in terms of creating such content, the closest thing I can think of is 3D editors, like Maya or Houdini, especially houdini( [link] ), which is the brilliant combination of computer graphics and programming, that allows you to model and program dynamic things visually.
Example of VR plus mocap manipulating a virtual space
Video list and Hacker News comment labels from HackerTube (prompted by @karpathy), released under ODC-By 1.0, built on the open-index/hacker-news archive. Hacker News data is provided by Y Combinator; comments remain the work of their authors. Video titles, channel names and thumbnails belong to their creators and are shown via YouTube. HackerTube Watch is an unofficial edition. Data export of 2026-10-31.
3 Hacker News comments
From the HN archive; up to 20 kept per video, favoring different commenters. Recommendations first.
Read more