I made this, fun to see my project on the front page!
The main question I tend to get from hacker-types is "what are the tech specs?". The demos run on a Mac with custom software I built for projection mapping/rendering/etc. (the hand tracking uses Apple's built-in framework, which is very good), a small consumer 4k laser projector, and a basic webcam for vision.
But really, this is primarily a research/design prototype - I think it's much more important to figure out what the user experience should ideally feel like, if the interactions are even plausible & desirable in the first place; and then work backwards to the technology.
Right. Tiny projectors have been done before.
Samsung had a smartphone with a projector back in 2014, but nobody wanted it.
Some startup called Moviphone has announced a new one for 2027.[1]
The use cases have not been compelling and fighting ambient lighting with a small projector is always tough.
Possible use case: projecting menus, ads, and bills on restaurant tables. With a camera and enough automated layout to work around what's on the table.
It might sell to the people who buy refrigerators with displays on the door.
Could be useful as a cooking training aid, like cooking videos but present where you need them.
The light bulb form factor is cute but probably not that useful. Most of the light will be aimed in non-useful directions. Look into ultra short throw projectors.
I met @heliographe a few years ago in a HN Tokyo (Slack) meeting. Happy to see you are still designing awesome things after your time at Apple. I look forward to seeing how your project(s) develop. PS: as soon as I saw the kitchen I knew this was Japan, and then I saw the hair and though —“hmm, I feel like I have seen this person before”— and it was you!!
Is a basic webcam really able to read text at that angle for the “what is this word” example?
I wasn’t sure if any of this was real and that example looked extremely fake to me because I had a hard time believing the device being able to read text from that distance.
I don't know about these things. To me they just turn everything into a screen.
Sure, it's a little more hands free, and projecting around definitly is nicer than screens. But projecting has a lot of problems too. Shadows, projection surface, ambiant light, heat are difficult to handle, but to me the worst is that it's not persistant. You can't just project 24/7 your family planner on the wall.
I'd really rather have something that prints, or better draws.
You want a recipe => print it, don't beam it.
You want a map of kyoto => tiny robot walks around the page and draws it for you. And you can draw over it if you want.
And it doesn't have to be exclusive, you can do both beam and print. But I think protable printing / drawing has a lot of unexplored potential.
Nice. This reminds me how I felt watching the first pre-iPhone multitouch demos.
One small quality-of-life improvement that could be added: have the software timestamp when you say "here" and then (using a short low-FPS video buffer?) analyze the corresponding frame of video. This would eliminate the pause when you're pointing at something, waiting for the computer to respond.
Alternately a low-latency speech analyzer could look for the word "here" and quickly snap a screenshot. Perhaps project a temporary dot on the surface, so the user knows they can put their hand down now?
Quite the impressive demo! There's huge potential here.
An interesting concept. I'm (probably) one of the few people that thought the Humane Pin had an interesting idea at its core, it was just executed poorly at the wrong time. A stationary version that leans into ambience solves a lot of the interaction and engineering issues.
I also like that it addresses my gripe with XR interfaces; you aren't actually sharing a digital object in space and it looks weird to everyone else.
Most of the videos are shot in real life, during summer daylight (only exception are the 2 person around a table ones, which are shot at night in a room with regular ceiling lighting).
And the projector I used is only 1100 ISO lumens; fancier models go way higher.
If you project mostly white content, it's remarkably readable + visible. Outdoors in direct sunlight would be another story, of course.
I realize this is not the makers' focus, but I find myself less enthused with the human-computer interaction side and more anxious about how we can ensure the human maintains true ownership and agency over the device and its software.
I mean, merge one of these with a modern anti-consumer "Smart TV", and we're uncomfortably close to 1984's telescreen.
I totally agree. It feels like we’ve crossed a line where tech is overtly hostile to users because everyone is chasing recurring revenue and data mining is the only viable option
Author here; I was definitely inspired by my visits to Dynamicland many years ago, which Folk Computer is also inspired by.
One big difference in direction is that Dynamicland + Folk Computer use projector+computer vision as substrate to try and reinvent/rethink the interactive/dynamic medium, and what end-user programming could be (which is very cool).
In that way I am much less ambitious; this exploration is more about how there is an entire class of existing interactions (the ones showed in the piece) that we perform today on screens, that could be much more interesting + collaborative if overlaid in the real world. And I am also terribly uninspired by the eyewear-based computing vision all major companies have pushed lately.
I think the mission statement here is on to something. Helmets, obtrusive physical appendages etc are unappealing to me. The idea of using extant space or surfaces however, is extremely appealing. And while it might suggest lower quality today, as with most things, it would almost certainly improve over iterations and time.
Holographic computing, I think, will be the future. At least for interfaces.
The main question I tend to get from hacker-types is "what are the tech specs?". The demos run on a Mac with custom software I built for projection mapping/rendering/etc. (the hand tracking uses Apple's built-in framework, which is very good), a small consumer 4k laser projector, and a basic webcam for vision.
But really, this is primarily a research/design prototype - I think it's much more important to figure out what the user experience should ideally feel like, if the interactions are even plausible & desirable in the first place; and then work backwards to the technology.
Possible use case: projecting menus, ads, and bills on restaurant tables. With a camera and enough automated layout to work around what's on the table.
It might sell to the people who buy refrigerators with displays on the door. Could be useful as a cooking training aid, like cooking videos but present where you need them.
The light bulb form factor is cute but probably not that useful. Most of the light will be aimed in non-useful directions. Look into ultra short throw projectors.
[1] https://moviphones.com/
I wasn’t sure if any of this was real and that example looked extremely fake to me because I had a hard time believing the device being able to read text from that distance.
Sure, it's a little more hands free, and projecting around definitly is nicer than screens. But projecting has a lot of problems too. Shadows, projection surface, ambiant light, heat are difficult to handle, but to me the worst is that it's not persistant. You can't just project 24/7 your family planner on the wall.
I'd really rather have something that prints, or better draws.
You want a recipe => print it, don't beam it.
You want a map of kyoto => tiny robot walks around the page and draws it for you. And you can draw over it if you want.
And it doesn't have to be exclusive, you can do both beam and print. But I think protable printing / drawing has a lot of unexplored potential.
One small quality-of-life improvement that could be added: have the software timestamp when you say "here" and then (using a short low-FPS video buffer?) analyze the corresponding frame of video. This would eliminate the pause when you're pointing at something, waiting for the computer to respond.
Alternately a low-latency speech analyzer could look for the word "here" and quickly snap a screenshot. Perhaps project a temporary dot on the surface, so the user knows they can put their hand down now?
Quite the impressive demo! There's huge potential here.
Reminds me of the projection bracelet scam that Captain Disillusion debunked years ago https://youtu.be/Cw7g8ixbomU?t=75
Most of the videos are shot in real life, during summer daylight (only exception are the 2 person around a table ones, which are shot at night in a room with regular ceiling lighting).
And the projector I used is only 1100 ISO lumens; fancier models go way higher.
If you project mostly white content, it's remarkably readable + visible. Outdoors in direct sunlight would be another story, of course.
I mean, merge one of these with a modern anti-consumer "Smart TV", and we're uncomfortably close to 1984's telescreen.
One big difference in direction is that Dynamicland + Folk Computer use projector+computer vision as substrate to try and reinvent/rethink the interactive/dynamic medium, and what end-user programming could be (which is very cool).
In that way I am much less ambitious; this exploration is more about how there is an entire class of existing interactions (the ones showed in the piece) that we perform today on screens, that could be much more interesting + collaborative if overlaid in the real world. And I am also terribly uninspired by the eyewear-based computing vision all major companies have pushed lately.
Youtube videos - https://www.youtube.com/results?search_query=sixth+sense+tec...
Holographic computing, I think, will be the future. At least for interfaces.