Kinect with Linux
The Minority Report has been in rotation on cable lately, and you've probably seen the futuristic vision of Tom Cruise standing in front of a large screen, manipulating information with waves of his hands. That vision is a bit closer to reality, thanks in part to the economies of scale of the game industry. I don't often have reason to sing the praises of Microsoft, particularly not in a magazine devoted to Linux and all things open. But one thing our friends in Redmond do very well is to commoditize hardware. They've done just that with the Kinect by creating it as a natural interface for the Xbox 360 game console. What's more, they've allowed open-source developers to create drivers for the device, and they've even allowed the third party who developed the technology, PrimeSense, Inc., to release its own device drivers for Linux, Windows and OS X.
Figure 1. The Author, as Seen in Kinect's Depth View
Computer interfaces that use "natural" interaction have been termed, appropriately, natural user interfaces, or NUI. The implication is not so much that the interface itself is natural (let's face it, there is no natural, unlearned way to interact with a machine), but that the interface is very easy to learn, and in that sense natural. Natural user interfaces are a very active research topic, and a wide variety of prototypes have been demonstrated. The list of projects exploring natural interfaces and Kinect is ever-expanding. Here are some that have been ventured on the Net:
Robotic vision systems.
Interior room mapping.
Light control with gestures.
Sign-language computer interfaces.
Standardized gesture interfaces.
Music with gestures (including the floor piano from Toys).
Instruction (such as dance, karate and tai chi).
The Kinect—Sensor Heaven
The Kinect is tightly packed with an array of sensors and specialized devices to preprocess the information received. Communication between the Kinect and the game console or Linux is through a single USB cable.
Several different estimates have been made of the cost to manufacture Kinect, ranging from about $56 to $150. Whatever the real cost, we have to hand it to Microsoft for reducing the cost of its original prototype (reportedly $30,000) by at least two orders of magnitude to create a viable commercial product.
You can view a complete teardown of the Kinect on YouTube (see Resources), and inside you will find the following:
A projector that projects a field of infrared beams, used to detect depth.
Two cameras (each 640x480): a monochrome infrared camera that is used to detect the array of reflected infrared beams (this is the information used to construct the image of depth) and a color camera that can be used to capture snapshots or as a Webcam.
A motor to tilt the sensor array up and down for best view of the scene.
An accelerometer to sense position.
Camera interfaces and preprocessors for the two cameras.
512MB of DDR memory and 8MB of Flash.
The infrared beams are encoded, and as they are reflected off surfaces in front of the sensor array and detected by the infrared camera, a preprocessor in the Kinect calculates the distance from the array to the reflecting surfaces. Proprietary software can use that information to identify likely humans in the scene and the likely positions of their arms and legs.
Kinect Open Source
The short history of the Kinect and open source is interesting, instructive, and it demonstrates the awesome power of open source:
On November 4, 2010, Microsoft released the Kinect in the United States.
That same day, Adafruit Industries announced it would pay anyone $1,000 for an open-source driver for the device on Windows or any other operating system.
Hours later, Microsoft announced that it would not condone hacking of any of its devices. Adafruit responded by raising the bounty to $2,000.
By Saturday, November 6, a hacker going by the name AlexP claimed to have hacked the interface to the Kinect's motor. Microsoft said it wasn't true. Adafruit raised the reward to $3,000.
By that Monday, AlexP had posted video proving his ability not only to control the motor, but also to interface to the depth perception and color camera on the Kinect. He could have chosen to release the code and collect the prize, but instead, that Tuesday, he posted a message saying he would release the code if $10,000 were contributed to fund his further work.
Tuesday evening, Adafruit released a large dataset recorded by a USB analyzer watching the data stream between the Kinect and Xbox. Hackers worldwide started using the dataset to work out the details of the interface.
On Wednesday, the Kinect was released in Europe. Hector Martin, near Bilbao, Spain, purchased one that morning, and using the Adafruit published data, was able to get it connected to his PC by noon. The results were published on the libfreenect site, and the prize was won.
But the story doesn't stop there. By Friday of the same week, Microsoft reconsidered its position on the open-source drivers. In a remarkable bit of semantic derring-do, Microsoft said the Kinect had not been hacked by its definition, and actually praised the developers expanding the use of the device.
In just a few days, the Open Source community had seized the opportunity provided by Microsoft and created an open-source alternative that created an explosion of possibilities for research into natural interfaces.
Rick Rogers has been a professional embedded developer for more than 30 years. Now specializing in mobile application software, when Rick isn't writing software for a living, he's writing books and magazine articles like this one.
Fast/Flexible Linux OS Recovery
On Demand Now
In this live one-hour webinar, learn how to enhance your existing backup strategies for complete disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible full-system recovery solution for UNIX and Linux systems.
Join Linux Journal's Shawn Powers and David Huffman, President/CEO, Storix, Inc.
Free to Linux Journal readers.Register Now!
- Download "Linux Management with Red Hat Satellite: Measuring Business Impact and ROI"
- Petros Koutoupis' RapidDisk
- The Italian Army Switches to LibreOffice
- Linux Mint 18
- ServersCheck's Thermal Imaging Camera Sensor
- Oracle vs. Google: Round 2
- The FBI and the Mozilla Foundation Lock Horns over Known Security Hole
- Varnish Software's Varnish Massive Storage Engine
- Privacy and the New Math
Until recently, IBM’s Power Platform was looked upon as being the system that hosted IBM’s flavor of UNIX and proprietary operating system called IBM i. These servers often are found in medium-size businesses running ERP, CRM and financials for on-premise customers. By enabling the Power platform to run the Linux OS, IBM now has positioned Power to be the platform of choice for those already running Linux that are facing scalability issues, especially customers looking at analytics, big data or cloud computing.
￼Running Linux on IBM’s Power hardware offers some obvious benefits, including improved processing speed and memory bandwidth, inherent security, and simpler deployment and management. But if you look beyond the impressive architecture, you’ll also find an open ecosystem that has given rise to a strong, innovative community, as well as an inventory of system and network management applications that really help leverage the benefits offered by running Linux on Power.Get the Guide