★ Simple Tricks and Nonsense
Setting aside potential trademark complaints from their friends at Disney, *this* is what Apple should have saved the term “Force Touch” for.
Upon launching Apple’s Encounter Dinosaurs app for Vision Pro — an immersive 3D experience — you see a butterfly a few feet in front of you. As it flutters about the space in front of you, if you reach out your hand, the butterfly will land on your extended finger. When I first experienced this demo back in June at WWDC, reps from Apple said — half boasting, half warning (both halves well-warranted) — that when the butterfly lands on your finger, some people actually feel it. They feel a physical tickle on their finger. I didn’t feel that, but some of my colleagues in the media who got the same demo did. They perceived a physical sensation from what is, ultimately, an optical illusion.
When an illusion is realistic enough, your brain accepts it as real, even though it’s not — and even though you know it’s not.
One generally underappreciated aspect of the iPhone is the accuracy of its finger-tracking for touch. From the beginning — the original iPhone on the day it launched — when you slide your finger on an iPhone to scroll, the scrolling tracks your finger movement exactly as though you were sliding a piece of paper. The delight began right from “Slide to Unlock”, because the knob you slid to unlock the iPhone tracked your finger precisely. It didn’t feel like you were issuing an indirect shortcut gesture; it felt (and to this day, of course, still feels) like you were directly manipulating whatever it was you were sliding or scrolling on screen. The bouncing when you scroll to the end of a view. The stretching when you pull down from the top of a view. It felt (and feels) real, to some degree.1
The experience of manipulating the user interface in VisionOS conveys something like this too. It’s not direct manipulation, though. You don’t reach out and grab a window bar to move the window. You just look the bar under the window, pinch your finger and thumb, and move your hand to move the window. It is as easy to pull a window toward you, or push it away, as it is to move it side to side. The same is true for resizing windows from the grabber that appears when you look at any corner, and it’s also true for using two hands — both making the finger-thumb pinch — to zoom the content (like, say, a photo) within a window. There’s no lag and there is tremendous precision. You can arrange, position, stack, and resize windows exactly the way you want them.
One comparison would be to a maestro conducting an orchestra. But that doesn’t convey the sense of precision in VisionOS — the sense of fine control through indirect manipulation. What it feels like is using the Force.
One of the first times we see any character use the Force in Star Wars2 is when Obi-Wan Kenobi uses the Jedi mind trick to convince the droid-hunting stormtroopers on the outskirts of Mos Eisley that R2-D2 and C-3PO are not, in fact, the droids they’re looking for. He even pinches his finger and thumb together while gesturing with his hand. It couldn’t be more apt an image.
In the very scene preceding Kenobi’s mind trick, we see Darth Vader using the Force to choke his vainglorious colleague Admiral Motti, with … a pinching gesture: “I find your lack of faith disturbing.”
Kenobi, conjuring an audio distraction to escape the attention of two stormtroopers:
George Lucas didn’t conceive of telekinesis, but his portrayal of it made it seem real, and defined how my and subsequent generations imagined they would invoke such an ability if they could. And now, in a sense — as hokey as this sounds — I feel like I can. Pulling virtual application windows through each other. Pushing them back against the wall. Expanding them. Shrinking them. Moving one outside my living room through a real glass window. Activating buttons and flipping switches just by looking at them, and gesturing.
And, like the butterfly from Encounter Dinosaurs that some people can feel landing on their outstretched finger, I swear to you, I can almost feel the telekinetic connection with UI elements in VisionOS. It’s a hint, a whiff, of tension — between not just my hands and the virtual elements I’m manipulating, but between my mind and those elements. Just the vaguest sensation of tension emanating from my forehead, like a taut thread of ultrafine string connecting my mind to the window I’m moving, or button I’m pressing, or photo I’m stretching.
With Vision Pro, the gestures are necessary, because the device can’t actually read your mind. (At least not in this first-generation version.) In Star Wars, the gestures exist not because they’re required to perform mystical feats of mentalism, but to make those actions cinematic. It is utterly natural to express ourselves via our hands, and to read meaning from the hand gestures of those we see.4 As good a job as the original film does conveying this, the most evocative shot exemplifying the use of a pinching gesture to convey telekinetic Force ability comes from 2016’s Rogue One, when Vader — who, whatever his flaws, never loses his sense of humor — offers Orson Krennic, father of the Death Star, this helpful advice: “Be careful not to choke on your aspirations, director.”
Setting aside potential trademark complaints from their friends at Disney, this is what Apple should have saved the term “Force Touch” for.
Android phones still don’t have this effect. It’s not because they don’t have high-enough frame rates or low-enough latency for touch input. It’s just that the math is off for putting everything together to create the illusion of direct manipulation. The physics aren’t quite right. It’s no longer about hardware specs (although that was a factor working against Android in the early years). It’s about craftsmanship. ↩︎
You want to get Star Wars nerdy? I’ll get Star Wars nerdy. I’ve thought about this stuff since I was in kindergarten. The first Force-capable character we see in Star Wars is Vader, leading the assault team that boards and commandeers Princess Leia’s ship, the Tantive IV. But it’s unclear whether he uses the Force even once in that entire scene. The one maybe is when he lifts a rebel officer off his feet by his neck, chokes him to death, then throws the body against a wall. In hindsight, it’s reasonable to suspect that Vader augments his physical strength with the Force while committing that murder, but as presented, the implication is only that he’s a physical beast.
When first we meet Kenobi on Tatooine, after Luke has been knocked unconscious by Tusken Raiders (a.k.a. Sand People), Kenobi scares them off by mimicking the roar of a krayt dragon. I’ve always presumed that was the Force, but that’s never made clear, either in the film or the screenplay. Kenobi then gently touches Luke’s forehead, perhaps using the Force to wake him from his concussed slumber. But again, it’s unclear if Kenobi uses the Force there, or if he’s simply checking Luke’s vitals and Luke just happens to wake. The screenplay simply states, “Ben puts his hand on Luke’s forehead and he begins to come around.”
Later in the film, in the above-pictured scene on the Death Star where Kenobi disables the tractor beam, he finds himself in need of a distraction to escape the attention of two stormtroopers. The screenplay reads:3
Ben moves around the tractor beam, watching the stormtroopers as
they turn their backs to him. Ben gestures with his hand toward
them, as the troops think they hear something in the other
hallway. With the help of the Force, Ben deftly slips past the
troopers and into the main hallway.
Again with the hand gesture — and confirmation that the Force can be used to create sounds. ↩︎︎
Speaking of deft, the revelation of the existence of the Force is wonderfully paced and executed in Star Wars. There’s no mention of “the Force”, nor do we see it in action (simulated krayt dragon roar aside), until over 30 minutes into the movie. The first half hour is spent establishing things like, “Wow, there are realistic-looking, cool-sounding space ships and laser guns and robots!”
Then comes the scene in Kenobi’s bachelor pad, where he tells — but doesn’t show — Luke about the Force, about the Jedi order, about their destruction at the hands of the Empire, and he gives him his (supposedly-dead) father’s lightsaber, which Luke, gadget hound that he is, of course immediately ignites and starts waving around. That scene is immediately followed by the one on the Death Star where we see Vader use the Force to choke Admiral Motti. Two short scenes later comes Obi-Wan’s “These aren’t the droids you’re looking for” mind trick. We’re told about the Force, see Vader wield it, then Kenobi. And boom: an already-mind-blowing movie has expanded from science fiction to fantasy. And in the midst of that sequence, we learn of a weapon that makes those laser guns no longer seem like the coolest things we’ve ever seen — or heard.
That the Force is used so sparingly throughout Star Wars is an immense part of the film’s gritty, grounded reality, and thus its charm. Amazing technology is everywhere, but much of it is grimy and old. The Force is almost never seen. Compare and contrast with many of the subsequent films, where the Force is used willy-nilly, like bubble gum in a baseball dugout, contributing to those films’ plasticky, video-game-like artificiality. ↩︎︎
We Philadelphians excel at communicating deep thoughts via our hands. ↩︎︎