The 19th Century Meets the 21st
I live at the foot of the Brooklyn Bridge, in a 150-year-old building that used to be the headquarters of the Brooklyn Railroad. Before the bridge was built, barges and ferries docked along the piers. Trains brought people and goods into Brooklyn and beyond. The bridge killed the ferries, the railroad and, eventually, the neighborhood—the heart of Brooklyn in Walt Whitman's day.
After a century of decline, the neighborhood, Fulton Ferry Landing, is being reclaimed by artists and people like me who work anywhere, thanks to the Internet. Disconnected for a hundred years, due to a network of roads that ignored it, Fulton Ferry Landing changed little. Today, it is quickly being reconnected to the rest of the planet, as a result of the most efficient transportation network ever devised.
Wall Street—just across the river—is one of the most wired neighborhoods on the planet. The rest of New York, however, lags behind most of North America in terms of connectivity options: few cable modems, little DSL, and ISDN being more expensive than digital leased lines. Most people make do with a modem. After seven years of dial-up access, I decided I could no longer wait for the dissolution of our local telephone and cable monopolies—it was taking too long.
Thanks to Linux, some volunteer work and a bit of luck, the old headquarters of the Brooklyn Railroad is now one of the most wired old buildings on the planet. Each apartment has two data ports connected to a router in the basement. The router is connected to the Internet through a digital leased line. We now have high-speed connectivity, 24 hours a day, for less than it used to cost us to maintain dial-up accounts.
A few months ago, I asked the building's residents to let me wire the building with CAT5 in order to set up a building network, because it made sense both economically and organizationally. Sharing resources, after all, is the whole point of packet switched networking—not wiring an apartment building is perverse.
Three years ago, I used to walk into many businesses that had each desktop computer connected to the Internet through a dial-up connection. Today, most of those businesses have connected their entire intranet to the Internet though a single, large pipe. Dial-up connections are expensive and inefficient; no IT organization would dream of setting things up that way. Apartment building dwellers, however, have barely begun to question the way they approach the problem. Although they enjoy high-speed, permanent connectivity at work, they don't question the AOL dial-up ritual at home.
To date, no one is offering residential network management service. If you want a building network, you will need an on-site expert to set up and maintain it.
In the U.S., the local telephone company is responsible for wires to the building. Residents are responsible for wires in their apartment. The building owner is responsible for the wiring within the rest of the building. I suspect it will take at least another two to three years for people to realize that IP is as fundamental as telephone service. At that point, they will start making noise about wiring capable of carrying data from the basement to their apartments. Network equipment vendors will start building and pricing hardware for this market, and of course, residential network management companies will be formed. In the meantime, we have to plan and build everything ourselves.
My building was not prepared to provide a network infrastructure. I guessed that trying to convince a majority of the shareholders to do so would be a waste of time, so I offered to pay for it all and resell the service to anyone who wanted it. Everyone with a computer has joined. While I would have preferred not to absorb all up-front costs, I have enjoyed the privilege of making decisions without running them by a committee.
The most significant up-front costs are the wiring, the router and the computer providing name, mail and web services. Running CAT5 wires from each apartment to a central hub—in our case, the basement—is never going to be cheap. In most cases, however, it will cost far less than it cost me. The age of the building worked against me.
In the early 20th century, services were run as if they would never require replacement. Electrical wires were buried in plaster walls. Telephones were wired directly to the building's exterior. Telephone jacks were a 1950s innovation, an early example of plug-and-play. Today, architects frequently design electrical systems to be accessible without the help of a demolition crew. Those who are truly forward-thinking will design easily accessible, parallel conduits: one for electricity and one for data. Today, “data” usually consists of telephone and cable television wires. Tomorrow, those two will be joined by computer network wires, which soon enough will also carry telephone and television data.
In our building, nothing is straightforward. Throughout the years, conduits have been run through the wood and concrete floors to carry electrical, telephone, intercom and cable TV wiring. None were large enough to accommodate additional wires. Running a new conduit was estimated to cost almost $1000 per apartment. That expense was impossible to justify at that time.
While I was mulling over what to do about this network wiring problem, another arose. The building ran out of telephone wires. Whoever did the capacity planning when the central wires were installed never considered fax lines, dial-up lines and two or three voice lines per unit. Also, the wires were old—many broke due to corrosion and many were static-filled. Clearly, I had another project on my hands.
Actually, I was lucky the building reached the end of its telephone network lifetime when it did. Any earlier, and I would not have had the foresight to run network lines in parallel with the new telephone lines. Any later, and I would probably have invested in a high-speed solution for myself and would not even have considered doing the work on a communal scale. The incremental cost of running the network wires was negligible, so I decided to go ahead and do it.
The great irony, of course, is that everyone has now canceled the lines they had for their dial-up service. Under the old system, we would now have plenty of lines.
Besides the wiring, the router and central computer turned out to be the other big cost in this sort of operation.
The router is expensive because each apartment needs its own subnet. I asked Cisco what they sold that could do the job. They literally answered that buying a router from them would cost me “both arms and both legs”. They did, however, suggest a “cheap” alternative: a low-end router and a switch, a solution that would have cost me about $3,500. I was not willing to spend half that much to solve this piece of the puzzle. I was fairly sure I could build what I needed using Linux.
My neighbor, a Linux guru, assured me I could. Before long, he and I had done the research and mapped out a strategy that worked.
The hardware we needed was free. Businesses all over town have mountains of 486s gathering dust in their storerooms. They are thrilled to give them away! As you know, the operating system we decided to use was also freely available.
Since our router was going to be a general purpose computer, we decided to run all of the shared services on the same computer. This simplified a lot of management issues. It also made disaster recovery relatively straightforward. We built a second, identical machine that can be swapped in for the first at a moment's notice. This sort of approach is practical only if a single machine is involved.
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
|Introduction to MapReduce with Hadoop on Linux||Jun 05, 2013|
- Containers—Not Virtual Machines—Are the Future Cloud
- Non-Linux FOSS: libnotify, OS X Style
- Linux Systems Administrator
- Validate an E-Mail Address with PHP, the Right Way
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Introduction to MapReduce with Hadoop on Linux
- RSS Feeds
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?