EOF - Turning the Internet Outside In
Doc Searls is Senior Editor of Linux Journal
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Sponsored by AMD
Built-in forensics, incident response, and security with Red Hat Enterprise Linux 6
Every security policy provides guidance and requirements for ensuring adequate protection of information and data, as well as high-level technical and administrative security requirements for a system in a given environment. Traditionally, providing security for a system focuses on the confidentiality of the information on it. However, protecting the data integrity and system and data availability is just as important. For example, when processing United States intelligence information, there are three attributes that require protection: confidentiality, integrity, and availability.
Learn more about catching the bad guy in this free white paper.
Sponsored by DLT Solutions
| Designing Electronics with Linux | May 22, 2013 |
| Dynamic DNS—an Object Lesson in Problem Solving | May 21, 2013 |
| Using Salt Stack and Vagrant for Drupal Development | May 20, 2013 |
| Making Linux and Android Get Along (It's Not as Hard as It Sounds) | May 16, 2013 |
| Drupal Is a Framework: Why Everyone Needs to Understand This | May 15, 2013 |
| Home, My Backup Data Center | May 13, 2013 |
- New Products
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Web & UI Developer (JavaScript & j Query)
- Designing Electronics with Linux
- Dynamic DNS—an Object Lesson in Problem Solving
- Using Salt Stack and Vagrant for Drupal Development
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Nice article, thanks for the
6 hours 48 min ago - I once had a better way I
12 hours 34 min ago - Not only you I too assumed
12 hours 51 min ago - another very interesting
14 hours 44 min ago - Reply to comment | Linux Journal
16 hours 38 min ago - Reply to comment | Linux Journal
23 hours 32 min ago - Reply to comment | Linux Journal
23 hours 48 min ago - Favorite (and easily brute-forced) pw's
1 day 1 hour ago - Have you tried Boxen? It's a
1 day 7 hours ago - seo services in india
1 day 12 hours ago
Enter to Win an Adafruit Pi Cobbler Breakout Kit for Raspberry Pi

It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Pi Cobbler Breakout Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- 5-21-13, Prototyping Pi Plate Kit: Philip Kirby
- Next winner announced on 5-27-13!
Featured Jobs
| Linux Systems Administrator | Houston and Austin, Texas | Host Gator |
| Senior Perl Developer | Austin, Texas | Host Gator |
| Technical Support Rep | Houston and Austin, Texas | Host Gator |
| UX Designer | Austin, Texas | Host Gator |
| Web & UI Developer (JavaScript & j Query) | Austin, Texas | Host Gator |
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?




Comments
Hacking from the Outside In
I've recently done quite a bit of hacking from the "ends" of the internet back inside. And I'm very very excited about it. Having noone really to share this with, I figured I would post it here.
At home my primary workstation runs Slackware 13 and can run a multitude of virtual guest machines. I've built it for that reason. I also have a DSL connection to the internet, nothing fancy, 8mb/512kb and a static IP. I also have a few IPv6 tunnels from HE.net.
My goal was to run virtual machines with private IPv4 addresses but public IPv6 addresses. Today that goal is a reality. There's a project called vde2 (Virtual Distributed Ethernet) from a group called VirtualSquare. To me, an understanding of internet protocols and this project were crucial. This is a virtual switch. A virtual switch which can also connect to other virtual switches over the internet.
So, on my server at home, I run a virtual machine. It gets an IPv4 address of 10.22.22.0/24 and can connect through to the regular internet via nat. It's also running radvd and handing out IPv6 addresses, but public ones. So every virtual machine I run gets a different public IPv6 address.
Now it gets even more interesting. I'm currently sitting at work. I can run a vde switch locally, and connect it (via ssh) like a virtual plug or crossover cable to home. Now if I run virtual machines (or even my host machine as this is a laptop), my local tap0 interface gets an IPv6 address... from home. As long as my ipv6 default gateway is correct, I'm connecting to the ipv6 internet using the ipv4 internet as a foundation. I connect through my office connection to home, then out to the ipv6 internet.
I can now run more than a dozen different OS's, from different physical locations, using the same IPv6 subnet from any physical location.
It was a bit of work on my part. But imagine wrapping this in a nice bundle and selling it as a service?
Internet hacking from the outside in, to go.
Come talk to me in #linuxjournal on freenode. I'm [flrichar] (n=fredr@2001:470:e075:2200:216:3eff:fe1e:eecb): Fred Richards, or [FredR](n=fredr@2001:470:e075:2200:d0ae:93ff:fe26:8415): Fred Richards ...
or someone@2001:470:e075:2200::0/64
I'm sure I'll run out of nicknames before I run out of IPs or virtual machines.
-- FLR or flrichar is a superfan of Linux Journal, and goofs around in the LJ IRC Channel
Simple Data Provider
As a Sr Network Engineer for an application data services company, I would _love_ to hack the internet from the ends inward. In fact, in theory, I guarantee I could build what you are looking for. I could build it in my head. But in reality, there are several hurdles to overcome.
Doc, you talk of home networking, but trust me, the typical consumer-grade networking equipment is pretty weak and limited. Networks are typically designed and built from the core out, with each piece of equipment having particular roles and responsibilities. From the core, out to the distribution level, then finally to the fringes of the access layer. Typically, access equipment could not handle the vast processing power necessary in the core.
Not saying that it couldn't be done, but an upgrade to end user and CPE equipment would be in order. Linux would take the same consumer hardware much, much further.
Another hurdle, would be to keep the historical players out of the loop. Why do the telcos and cablecos have a duopoly? Well, whenever anyone needed a wire to their home, they were the only ones in the business of running those cables. Long distance, high bandwidth wireless (802.16 Wimax perhaps) would be a better alternative, based on standards of course.
Another bad habit of ISPs is oversubscribing. Most companies are in business to make money, at any and all costs, to the point where I've seen some pretty crazy over-subscriptions. Your cable headend node probably serves upwards of 4000 people. Do you live in a highly populated area?
So at the CPE side, you want a powerful, wireless unit, with lots of diversity. With the ability to connect to many others, where the greater number of your neighbors would work in your favor instead of against it. That piece really needs the "bittorrent" effect. Think that can be done currently with ipv4? I don't think so.
Where are we going to get millions of IP addresses per end user? I'm sure IPv6 could fill that gap.
These are just some of my ideas. Instead of following the normal layers, from physical on up, I'm thinking more of the top down, solving issues with protocols first. It still needs some under laying infrastructure help to flesh out.
One of my dreams is to create a "data provider". A company who's sole purpose is to run data to your home. That's it. DNS? Email, web hosting? On your own. Supply only raw IP connectivity. The local water authority doesn't dictate how I use water in my home, so why is my ISP so involved in my data?
Just some thoughts.
-- FLR or flrichar is a superfan of Linux Journal, and goofs around in the LJ IRC Channel