Prosthetic Knowledge Picks: Kinect Genealogy - A Brief History of Gestural Interfaces

In honor of the approaching second birthday of the release of Microsoft's Kinect, we will take a brief tour of the experimental technology that preceded it. But before we begin, it is worth noting a few facts about the Kinect we know today, a piece of technology that almost overnight changed the development of contemporary interactive art by being powerful and affordable. 'Project Natal', as it was originally known, initially used a system called "Time-of-flight" which had origins military laser radar systems, but changed when a start-up called Primesense, an Israeli company made of ex-military engineers, were trying to sell their consumer-focused product:

the PrimeSense technology uses a proprietary technology called “light coding,” rather than the time-of-flight cameras used by of its competitors. Time-of-flight emits strong pulses of light and measures the delay in their return to calculate positions.
“Time-of-flight came from laser radar systems with military applications,” said Aviad and Inon. “[But] the DNA of the PrimeSense technology was from day one for the consumer market.
“There are a lot of differences between PrimeSense and time-of-flight cameras in general. PrimeSense has achieved a breakthrough on price and performance. The performance we generated through the device is better in a long list of parameters [than time-of-flight].”

PrimeSense's product was initially targeting Apple, a sensible approach to a company that introduced a new type of interface to mass market. Yet, it wasn't to be:

“It was the most natural place for the technology,” [Inon Beracha, CEO of PrimeSense] said.
Apple has a history of interface innovation, of course, and had recently introduced the iPhone with its paradigm-shifting multitouch UI. PrimeSense’s system went one step further: It was multitouch that you didn’t even have to touch. Apple seemed like a natural fit.
Yet the initial meetings hadn’t gone so well. Obsessed with secrecy, Apple had already asked Beracha to sign a stack of crippling legal agreements and NDAs.
He shook his head. Why didn’t he want to do a deal with Apple? No need. The technology was hot. He could sell it to anyone. “Apple is a pain in the ass,” he said, smiling.

A decision which certainly benefitted artists, as it would be likely that the technology would be 'locked-in' with Apple, and Microsoft's efforts would have taken longer to reach everyone (as well as possibly more expensive - it has been noted that PrimeScene sell their sensor circuit boards to Microsoft at $10 each.)

Now, let's introduce ourselves to previous gestural interfaces, some of which are surprisingly older than you initially think ....

ANIMAC (early 1960s)

This, admittedly is not a system to interact directly with computer information, and is more closer a relation to MOCAP technology. However, it relates to the Kinect in the sense that gestural information from a participant is used to direct the animated information output, in the same way as the Kinect's own skeletal detection system works:

Perhaps one of the earliest pioneers of this analog computer animation approach was Lee Harrison III. In the early 1960s, he experimented with animating figures using analog circuits and a cathode ray tube. Ahead of his time, he rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, called ANIMAC …

… It was while he was at Philco that he decided to chase his idea of systematically creating animated figures. His concept was to view a stick figure as a collection of lines that could be independently moved and positioned to form an animated character. Each of the lines would be displayed on a CRT and controlled with a vector deflection of the CRT’s electron beam. Each figure would be composed of bones, skin, joints, wrinkles, eyes, and moving lips, all drawn in sequence to create what Harrison called a “cathode ray marionette.”

Here is a video which features some of the visual output the machine was capable of (you will need to move to the one minute twenty mark):

Links: Section 12 - A Critical History of Computer Graphics and Animation
PK Link

Pantomation (1977 - 1979)

An early gestural graphic system which was initially designed for music scoring and performance art:

Pantomation was a very early tracking chromakey system from the 1970s. Originally intended for music scoring, the system was adapted to other styles of performance art. While crude by modern standards, the concept was decades ahead of its time; it can reasonably be considered an early forebear of systems like Microsoft's Kinect.

Source: VintageCG
PK Link

Videoplace (1985) by Myron Krueger

Another example via VintageCG, advancing with a much more digital and creative approach:

Videoplace, developed in the mid-1970s, but with work continuing through the late 1980s, was a pioneering creation of full-body interactivity and virtual environments. It contained 25 different environments (or interaction patterns) in which people could engage. He coined the term “Artificial Reality” to describe what he was striving to create.
His work is a clear precursor to such developments as multi-touch, the Sony Eye Toy (2003), and Microsoft’s Project Natal (2009).

Links: Original Inventing Interactive piece
Video Link
PK Link

GestureTek (1991)

Probably the biggest leap in gesture technology so far (to the point that the company owns many patents which are employed by the Kinect). Running on a Commodore Amiga 1000, here is a demonstration featuring colourful dancing, virtual musical instruments and Weird Al Yankovic!:

This is the future of Kinect Musical Performance. Vincent John Vincent invented video gesture control virtual reality with Francis MacDougall in 1986. By 1991 they had a fully dynamic immersive video gesture control virtual reality system. Vincent started performing around the world in 1986. He created a whole new genre of performance technology and hundreds of virtual performance instrument scenes, including musical scales, soundscapes, virtual drum kits, virtual juggling, virtual dance and a world of other creative landscapes. He is the President and Co-Founder of GestureTek, the worlds leading Video Gesture Control company.They and their company, Gesturetek, wented on to invent may more video gesture control products, including interactive surfaces and 3D video gesture control (2000), which is the technology behind the Kinect. GestureTek holds over 18 patents, and has licensed patents and technology to Microsoft for the XBOX, and to Sony for the PlayStation, etc. www.vjvincent.com ,www.gesturetek.com
Michael Lowcock wrote this Song " Far beyond the sighted mind".

Links: The Atlantic's piece on the technology
PK Link

Finally ...

What does the Kinect's 'Light Coding' technology look like? Here is a video using infra-red goggles to see the emissions which calculate the depth of what is in front of it: