Put Your Servers in the Cloud with Amazon EC2 and Ubuntu
Cloud services are all the rage today, although some of my fellow Linux Journal staffers may scoff when they hear me say that. Cloud services is a nebulous term that can mean anything from completely hosted services (like Gmail) to virtualized, leased servers, such as those provided by Amazon's EC2 service. And, the latter is the subject of this article. Recently, with the advent of Ubuntu 9.10 (Karmic Koala), Canonical has added support for pre-baked Amazon EC2 images. This makes spinning up your own personal cloud servers fast and easy—although not necessarily economical (see the Amazon EC2 Economics sidebar for a quick cost breakdown of EC2).
The Ubuntu EC2 Starters Guide (see Resources) should be your first stop. This document guides you through the process of creating your own EC2 instance. Before you can do anything at all with EC2, you need to set up an Amazon EC2 account. Go to the URL listed in the Resources section, and either sign in with your existing Amazon account or create a new one. Then, click on the EC2 link and sign up for the EC2 service. You need to provide a credit card for billing purposes. Once you've done that, select the Create an X.509 Certificate link that's presented on the thank-you page. Select yes to create a new certificate, and then download your certificate and private key to your hard disk. Finally, make a note of your AWS account ID number, as you may need it later.
Now that you've got your Amazon account and are enrolled in the EC2 program, it's time to start installing the management tools on your local system. If you're running Ubuntu, simply type sudo apt-get install ec2-api-tools to download the EC2 management tools. You also need the Sun 1.6 JDK (installable via sudo apt-get install sun-java6-jdk). If you're using another distro or Mac OS X, you can get the toolset by following the link in the Resources section of this article. The tools are nothing more than a .zip file full of Java binaries and shell scripts, so they're fairly portable.
SIGN UP for Linux Journal's monthly Cloud Computing newsletter - it's free!
Now that you've got the tools, the next step is to edit your environment variables. Simply adding the following lines to your ~/.bashrc file will do the trick (make sure to edit the path and filename to suit your own setup):
export EC2_PRIVATE_KEY=$HOME/To test the tools, run the following from a new terminal:
/pk-XX.pem export EC2_CERT=$HOME/ /cert-XX.pem export JAVA_HOME=/usr/lib/jvm/java-6-sun/jre
ec2-describe-images -o self -o amazonYou should receive a list of all the publicly available EC2 images published by Amazon.
Because the Ubuntu EC2 images are publicly available, you need to generate an SSH key to access them. Do this on your local machine by running the command:
ec2-add-keypair ec2-keypair > ec2-keypair.pemMake sure the permissions on the file are read-write for your user, with no access for anyone else (chmod 600 ec2-keypair.pem).
Now that you've finished the groundwork, you're ready to start up your first Ubuntu 9.10 instance in the cloud. The ec2-run-instances command will start your instance for you, but you need to feed it an AMI ID and your keypair. The AMI ID is nothing more than a unique identifier that is associated with a published EC2 image. Because I want to instantiate the “small” Ubuntu 9.10 EC2 image for this example, I use the “ami-52be5d3b” AMI ID. The links to publicly available Ubuntu 9.10 images and their AMI IDs are available in the Resources section of this article. Once you run the ec2-run-instances command, you will see output similar to the following:
$ ec2-run-instances ami-ab15f6c2 \ --key ec2-keypair --instance-type m1.small RESERVATION r-d8b376b0 748502897232 default INSTANCE i-bc9913d4 ami-ab15f6c2 \ pending ec2-keypair 0 \ m1.small 2009-11-02T22:23:12+0000 \ us-east-1d aki-76be5d1f ari-4cbe5d25Instances may take a few minutes to start up (shown by the “pending” status in the output above), particularly if this is your first one. To check on the status of your instance, you can run the ec2-describe-instances command. When your instance is running, you'll see something similar to the following:
$ ec2-describe-instances RESERVATION r-d8b376b0 748502897232 default INSTANCE i-bc9913d4 ami-ab15f6c2 \ ec2-72-44-62-167.compute-1.amazonaws.com \ domU-12-31-39-00-ED-A3.compute-1.internal \ running ec2-keypair 0 \ m1.small 2009-11-02T22:23:12+0000 \ us-east-1d aki-76be5d1f ari-4cbe5d25The “running” tag in the output denotes that the instance is up and running. Also listed is the Internet-facing DNS name as well as the internally facing Amazon DNS name. The default image has SSH running, but the EC2 firewall rules deny inbound port 22 access. To enable SSH access to the EC2 instance, you need to run ec2-authorize default -p 22. Once that is done, you can ssh to your instance using the SSH keys generated earlier:
$ ec2-authorize default -p 22 GROUP default PERMISSION default ALLOWS tcp 22 22 FROM CIDR 0.0.0.0/0 $ ssh -i ~/.ec2/ec2-keypair.pem \ email@example.com Linux domU-12-31-39-00-ED-A3 2.6.31-300-ec2 #3-Ubuntu SMP \ Sat Sep 26 10:31:44 UTC 2009 i686 To access official Ubuntu documentation, please visit: http://help.ubuntu.com/ System information as of Mon Nov 2 22:45:44 UTC 2009 System load: 0.0 Memory usage: 1% Processes: 56 Usage of /: 7.6% of 9.92GB Swap usage: 0% Users logged in: 0 ... ubuntu@domU-12-31-39-00-ED-A3:~$At this point, your instance is ready for you to start configuring whatever software you choose to run on it. Because it's essentially an Ubuntu machine, administration and package management is done just like on the systems you're used to (assuming you use Ubuntu). You simply can use apt-get!
As an example, I'm going to get a quick Apache server running. First, I update the apt indexes on the instance by running sudo apt-get update. Next, I install Apache on the instance by running sudo apt-get install apache2. Apache is installed and running using the default Ubuntu configuration. However, I can't actually hit the Web server from my desktop here, as port 80 and 443 are disallowed by the EC2 firewall. I do a quick ec2-authorize default -p 80, and now Firefox on my local machine can hit the Web server I just installed on the EC2 instance, as shown in Figure 1. It's not the most exciting of Web pages, but it's something!
Figure 1. My Web server's in the cloud!
Last but not least, you'll want to terminate or shut down your instances when they're not in use to save money. That's done via the ec2-terminate-instances command. Simply run it with the ID number of your instance (which can be determined via the ec2-describe-instances command), and your instance will terminate:
$ ec2-describe-instances RESERVATION r-d8b376b0 748502897232 default INSTANCE i-bc9913d4 ami-52be5d3b \ ec2-72-44-62-167.compute-1.amazonaws.com \ domU-12-31-39-00-ED-A3.compute-1.internal \ running ec2-keypair 0 \ m1.small 2009-11-02T22:23:12+0000 \ us-east-1d aki-76be5d1f ari-4cbe5d25 $ ec2-terminate-instances i-bc9913d4 INSTANCE i-bc9913d4 running shutting-downThere you have it. You can run your own servers “in the cloud”, thanks to Canonical and Amazon. It's certainly a lot faster than installing your own OS, and it requires no physical equipment nor the need to buy anything. EC2 instances can be a great way to extend your own computing platform, provide capacity on demand to a service or enable you to spin up a replacement server. Whatever you use EC2 for, keep in mind the hourly rate, or you run the risk of getting an unexpected bill at the end of the month.
Amazon EC2 Economics
Amazon's servers are priced on an hourly basis, for every hour the instance is running. The baseline server, otherwise known as the “small” configuration, is basically a Xen virtual machine with 1.7GB of RAM, one core of CPU measuring one EC2 unit (approximately a 1.7GHz CPU) and 160GB of storage. The “large” configuration is 7.5GB of RAM, two cores with two EC2 units each and 850GB of storage. The small instance is $0.085 per hour, and the large instance is $0.34 per hour. Amazon also charges for data sent and received. Incoming data is billed at $0.10 per GB, and outgoing data is billed at $0.17 per GB. Larger plans are available for more compute-intensive applications, but the cost for those is even higher. This means that the average small instance probably will run around $70 per month to operate, and the large instance will run somewhere around $250 per month. Of course, this also depends on the amount of bandwidth consumed. Although the monthly costs are a little on the pricey side, there are no setup fees, equipment maintenance costs or capital expenses needed to run this solution. EC2 isn't a one-size-fits-all solution, but it may make sense for some applications.
How Can I Figure My Costs, Exactly?
It's rather tough to nail down cost for an EC2 instance, as it depends on things like bandwidth. However, there is a small dashboard-like application you can install that can help you track your costs. The package byobu (formerly known as screen-profiles) can help. After apt-getting byobu and running byobu, you'll get a screen session with a small two-line dashboard at the bottom of your terminal window. To add the EC2 information, press the F9 key to bring up the byobu menu, select Toggle Status Notifications, then select the ec2_cost notifier, and press Apply. Then you'll have a neat little running total of the approximate cost for your EC2 instance, as shown in Figure 2. You can see this particular run cost me 40 cents! Byobu has all kinds of other useful little widgets too; check it out if you do a lot of management via a terminal.
Figure 2. Pennies Here, Pennies There
- Ubuntu EC2 Starters Guide: https://help.ubuntu.com/community/EC2StartersGuide
- Amazon AWS Portal: aws.amazon.com
- Amazon EC2 Signup: aws.amazon.com/ec2
- Amazon EC2 Getting Started Guide: docs.amazonwebservices.com/AWSEC2/latest/GettingStartedGuide
- Amazon EC2 API/Management Tools: developer.amazonwebservices.com/connect/entry.jspa?externalID=351&categoryID=88
- Ubuntu EC2 Image List: uec-images.ubuntu.com/releases/karmic/release
Bill Childers is the Virtual Editor for Linux Journal. No one really knows what that means.
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?
|Designing Electronics with Linux||May 22, 2013|
|Dynamic DNS—an Object Lesson in Problem Solving||May 21, 2013|
|Using Salt Stack and Vagrant for Drupal Development||May 20, 2013|
|Making Linux and Android Get Along (It's Not as Hard as It Sounds)||May 16, 2013|
|Drupal Is a Framework: Why Everyone Needs to Understand This||May 15, 2013|
|Home, My Backup Data Center||May 13, 2013|
- New Products
- Linux Systems Administrator
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- Designing Electronics with Linux
- Dynamic DNS—an Object Lesson in Problem Solving
- Using Salt Stack and Vagrant for Drupal Development
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- another very interesting
1 hour 5 min ago
- Reply to comment | Linux Journal
2 hours 58 min ago
- Reply to comment | Linux Journal
9 hours 52 min ago
- Reply to comment | Linux Journal
10 hours 9 min ago
- Favorite (and easily brute-forced) pw's
12 hours 17 sec ago
- Have you tried Boxen? It's a
17 hours 52 min ago
- seo services in india
22 hours 23 min ago
- For KDE install kio-mtp
22 hours 24 min ago
- Evernote is much more...
1 day 24 min ago
- Reply to comment | Linux Journal
1 day 9 hours ago