Kinect with Linux
The Minority Report has been in rotation on cable lately, and you've probably seen the futuristic vision of Tom Cruise standing in front of a large screen, manipulating information with waves of his hands. That vision is a bit closer to reality, thanks in part to the economies of scale of the game industry. I don't often have reason to sing the praises of Microsoft, particularly not in a magazine devoted to Linux and all things open. But one thing our friends in Redmond do very well is to commoditize hardware. They've done just that with the Kinect by creating it as a natural interface for the Xbox 360 game console. What's more, they've allowed open-source developers to create drivers for the device, and they've even allowed the third party who developed the technology, PrimeSense, Inc., to release its own device drivers for Linux, Windows and OS X.
Figure 1. The Author, as Seen in Kinect's Depth View
Computer interfaces that use "natural" interaction have been termed, appropriately, natural user interfaces, or NUI. The implication is not so much that the interface itself is natural (let's face it, there is no natural, unlearned way to interact with a machine), but that the interface is very easy to learn, and in that sense natural. Natural user interfaces are a very active research topic, and a wide variety of prototypes have been demonstrated. The list of projects exploring natural interfaces and Kinect is ever-expanding. Here are some that have been ventured on the Net:
Robotic vision systems.
Interior room mapping.
Light control with gestures.
Sign-language computer interfaces.
Standardized gesture interfaces.
Music with gestures (including the floor piano from Toys).
Instruction (such as dance, karate and tai chi).
The Kinect—Sensor Heaven
The Kinect is tightly packed with an array of sensors and specialized devices to preprocess the information received. Communication between the Kinect and the game console or Linux is through a single USB cable.
Several different estimates have been made of the cost to manufacture Kinect, ranging from about $56 to $150. Whatever the real cost, we have to hand it to Microsoft for reducing the cost of its original prototype (reportedly $30,000) by at least two orders of magnitude to create a viable commercial product.
You can view a complete teardown of the Kinect on YouTube (see Resources), and inside you will find the following:
A projector that projects a field of infrared beams, used to detect depth.
Two cameras (each 640x480): a monochrome infrared camera that is used to detect the array of reflected infrared beams (this is the information used to construct the image of depth) and a color camera that can be used to capture snapshots or as a Webcam.
A motor to tilt the sensor array up and down for best view of the scene.
An accelerometer to sense position.
Camera interfaces and preprocessors for the two cameras.
512MB of DDR memory and 8MB of Flash.
The infrared beams are encoded, and as they are reflected off surfaces in front of the sensor array and detected by the infrared camera, a preprocessor in the Kinect calculates the distance from the array to the reflecting surfaces. Proprietary software can use that information to identify likely humans in the scene and the likely positions of their arms and legs.
Kinect Open Source
The short history of the Kinect and open source is interesting, instructive, and it demonstrates the awesome power of open source:
On November 4, 2010, Microsoft released the Kinect in the United States.
That same day, Adafruit Industries announced it would pay anyone $1,000 for an open-source driver for the device on Windows or any other operating system.
Hours later, Microsoft announced that it would not condone hacking of any of its devices. Adafruit responded by raising the bounty to $2,000.
By Saturday, November 6, a hacker going by the name AlexP claimed to have hacked the interface to the Kinect's motor. Microsoft said it wasn't true. Adafruit raised the reward to $3,000.
By that Monday, AlexP had posted video proving his ability not only to control the motor, but also to interface to the depth perception and color camera on the Kinect. He could have chosen to release the code and collect the prize, but instead, that Tuesday, he posted a message saying he would release the code if $10,000 were contributed to fund his further work.
Tuesday evening, Adafruit released a large dataset recorded by a USB analyzer watching the data stream between the Kinect and Xbox. Hackers worldwide started using the dataset to work out the details of the interface.
On Wednesday, the Kinect was released in Europe. Hector Martin, near Bilbao, Spain, purchased one that morning, and using the Adafruit published data, was able to get it connected to his PC by noon. The results were published on the libfreenect site, and the prize was won.
But the story doesn't stop there. By Friday of the same week, Microsoft reconsidered its position on the open-source drivers. In a remarkable bit of semantic derring-do, Microsoft said the Kinect had not been hacked by its definition, and actually praised the developers expanding the use of the device.
In just a few days, the Open Source community had seized the opportunity provided by Microsoft and created an open-source alternative that created an explosion of possibilities for research into natural interfaces.
Rick Rogers has been a professional embedded developer for more than 30 years. Now specializing in mobile application software, when Rick isn't writing software for a living, he's writing books and magazine articles like this one.
|Updates from LinuxCon and ContainerCon, Toronto, August 2016||Aug 23, 2016|
|NVMe over Fabrics Support Coming to the Linux 4.8 Kernel||Aug 22, 2016|
|What I Wish I’d Known When I Was an Embedded Linux Newbie||Aug 18, 2016|
|Pandas||Aug 17, 2016|
|Juniper Systems' Geode||Aug 16, 2016|
|Analyzing Data||Aug 15, 2016|
- Updates from LinuxCon and ContainerCon, Toronto, August 2016
- NVMe over Fabrics Support Coming to the Linux 4.8 Kernel
- What I Wish I’d Known When I Was an Embedded Linux Newbie
- New Version of GParted
- Tech Tip: Really Simple HTTP Server with Python
- All about printf
- Analyzing Data
- Tor 0.2.8.6 Is Released
- Returning Values from Bash Functions
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide