Pass on Passwords with scp
In this article, I show you how to use the scp (secure copy) command without needing to use passwords. I then show you how to use this command in two scripts. One script lets you copy a file to multiple Linux boxes on your network, and the other allows you to back up all of your Linux boxes easily.
If you're a Linux sysadmin, you frequently need to copy files from one Linux box to another. Or, you may need to distribute a file to multiple boxes. You could use FTP, but using scp has many advantages. For instance, scp is much more secure than FTP. scp travels across the LAN/WAN encrypted, while FTP uses clear text, even for passwords.
But what I like best about scp is it's easily scriptable. Suppose you need to distribute a file to 100 Linux boxes. I'd rather write a script to do this than type 100 sets of copy commands. If you use FTP in your script, things can get messy, because each Linux box you log into is going to ask for a password. But if you use scp in your script, you can set things up so the remote Linux boxes don't ask for a password. Believe it or not, this actually is much more secure than using FTP.
Here's an example demonstrating the most basic syntax for scp. To copy a file named abc.tgz from your local PC to the /tmp dir of a remote PC called bozo, use:
scp abc.tgz root@bozo:/tmp
You now are asked for bozo's root password, so we're not quite there yet. The system still is asking for a password, so it's not easily scriptable. To fix that, follow this one-time procedure, after which you can make endless password-less scp copies:
Decide which user on the local machine will be using scp later on. Of course, root gives you the most power, and that's how I personally have done it. I'm not going to give you a lecture here on the dangers of root, so if you don't understand them, choose a different user. Whatever you choose, log in as that user now and stay there for the rest of the procedure. Log in as this same user when you use scp later on.
Generate a public/private key pair on the local machine. Say what? If you're not familiar with public key cryptography, here's the 15-second explanation. In public key cryptography, you generate a pair of mathematically related keys, one public and one private. You then give your public key to anyone and everyone in the world, but you never ever give out your private key. The magic is in the mathematical makeup of the keys; anyone with your public key can use it to encrypt a message, but only you can decrypt it with your private key. Anyway, the syntax to create the key pair is:
ssh-keygen -t rsa
In response, you should see:
Generating public/private rsa key pair Enter file in which to save the key ...
Press Enter to accept this.
In response, you should see:
Enter passphrase (empty for no passphrase):
You don't need a passphrase, so press Enter twice.
In response, you should see:
Your identification has been saved in ... Your public key has been saved in ...
Note the name and location of the public key just generated. It always ends in .pub.
Copy the public key just generated to all of your remote Linux boxes. You can use scp or FTP or whatever to make the copy. Assuming you're using root--again, see my warning in step 1--the key must be contained in the file /root/.ssh/authorized_keys. Or, if you are logging in as a user, for example, clyde, it would be in /home/clyde/authorized_keys. Notice that the authorized_keys file can contain keys from other PCs. So, if the file already exists and contains text, you need to append the contents of your public key file to what already is there.
Now, with a little luck, you should be able to scp a file to the remote box without needing to use a password. So let's test it by trying our first example again. Copy a file named xyz.tgz from your local PC to the /tmp dir of a remote PC called bozo:
scp xyz.tgz root@bozo:/tmp
Wow--it copied with no password!
A word about security before we go on. This local PC just became pretty powerful, as it now has access to all the remote PCs with only the one local password. So that one password better be strong and well guarded.
Now for the fun part. Let's write a short script to copy a file called houdini from the local PC to the /tmp dir of ten remote PCs, located in ten different cities, in only five minutes of work. Of course, it would work the same with 100 or 1000 of PCs. Suppose the 10 PCs are called brooklyn, oshkosh, paris, bejing, winslow, rio, gnome, miami, minsk and tokyo. Here's the script:
#!/bin/sh for CITY in brooklyn oshkosh paris bejing winslow rio gnome miami minsk tokyo do scp houdini root@$CITY:/tmp echo $CITY " is copied" done
It works like magic. With the echo line in this script, you should be able to watch as each city's copy is completed one after the next.
By the way, if you're new to shell scripting, here is a pretty good tutorial.
As you may know, scp is only one part of the much broader SSH program. Here's the cool part: when you followed my six-step procedure above, you also gained the ability to sit at your local PC and execute any command you like on any of the remote PCs--without a password, of course. Here's a simple example to view the date and time on the remote PC brooklyn:
ssh brooklyn "date"
Let's now put these two concepts together for one final and seriously cool script. It's a down-and-dirty way to back up all of your remote Linux boxes. The example backs up the /home dir on each box. It's primitive compared to the abilities of commercial backup software, but you can't beat the price. Consider the fact that most commercial backup software charges license fees for each machine you back up. If you use such a package, instead of paying license fees to back up 100 PCs remotely, you could use the script to back up the 100 PCs to one local PC. Then, back up the local PC to your commercial package and save the license fees for 99 PCs. Anyway, the script below demonstrates the concepts, so you can write your own script to suit your own situation. Simply put this script in a cron job on your local PC; no script is required on the remote PCs. Please read the comments carefully, as they explain everything you need to know:
#!/bin/sh # Variables are upper case for clarity # before using the script you need to create a dir called '/tmp/backups' on each # remote box & a dir called '/usr/backups' on the local box # on this local PC # Set the variable "DATE" & format the date cmd output to look pretty # DATE=$(date +%b%d) # this 'for loop' has 3 separate functions for CITY in brooklyn oshkosh paris bejing winslow rio gnome miami minsk tokyo do # remove tarball on remote box from the previous time the script ran # to avoid filling up your HD # then echo it for troubleshooting # ssh -1 $CITY "rm -f /tmp/backups/*.tgz" echo $CITY " old tarball removed" # create a tarball of the /home dir on each remote box & put it in /tmp/backups # name the tarball uniquely with the date & city name # ssh $CITY "tar -zcvpf /tmp/backups/$CITY.$DATE.tgz /home/" echo $CITY " is tarred" # copy the tarball just create from the remote box to the /usr/backups dir on # the local box # scp root@$CITY:/tmp/backups/$CITY.$DATE.tgz /usr/backups echo $CITY " is copied" done # the rest of the script is for error checking only, so it's optional: # on this local PC # create error file w todays date. # If any box doesn't get backed, it gets written to this file # touch /u01/backup/scp_error_$DATE for CITY in brooklyn oshkosh paris bejing winslow rio gnome miami minsk tokyo do # Check if tarball was copied to local box. If not write to error file # note the use of '||' which says do what's after it if what's before it is not # true # ls /u01/backup/$CITY.$DATE.tgz || echo " $CITY did not copy" >> scp_error_$DATE # Check if tarball can be opened w/o errors. If errors write to error file. tar ztvf /u01/backup/$CITY.$DATE.tgz || echo "tarball of $CITY is No Good" >> scp_error_$DATE done
That's about it. In this article, I've tried to give examples that demonstrate the concepts and are not necessarily ready to be used "as is". Some of the syntax may not work for all distributions, but in the interest of brevity, I could not include all the possibilities. For example, if you are using Red Hat 6.2 or before, the syntax requires some changes. So be creative, and hopefully you can use some of this work in your own environment.
Copyright (c) 2004, Dave Sirof. Originally published in Linux Gazette issue 98. Copyright (c) 2004, Specialized Systems Consultants, Inc.
Practical Task Scheduling Deployment
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.View Now!
|The Firebird Project's Firebird Relational Database||Jul 29, 2016|
|Stunnel Security for Oracle||Jul 28, 2016|
|SUSE LLC's SUSE Manager||Jul 21, 2016|
|My +1 Sword of Productivity||Jul 20, 2016|
|Non-Linux FOSS: Caffeine!||Jul 19, 2016|
|Murat Yener and Onur Dundar's Expert Android Studio (Wrox)||Jul 18, 2016|
- Stunnel Security for Oracle
- The Firebird Project's Firebird Relational Database
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- SUSE LLC's SUSE Manager
- Managing Linux Using Puppet
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- Google's SwiftShader Released
- SuperTuxKart 0.9.2 Released
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide