Tweaking Tux, Part 2
Now that I've got this time off thing out of my system, let's get back to some of the business at hand, namely tweaking old Tux for fun, performance and a better knowledge of the innards of your Linux system. In the last installment, I gave you some ideas for tuning network parameters in the /proc filesystem. For many /proc tweaks, it is simply a matter of writing the appropriate value in the appropriate file and great things happen. As a refresher, here is what we did to turn on IP forwarding, via /proc rather than at boot time or through other menus.
echo "1" > /proc/sys/net/ipv4/ip_forward
To make sure that these parameters take, we would add this line to our rc.local boot script and call it done. This brings me to a couple of questions I received from readers while I was away. Somebody e-mailed to tell me that they could not find rc.local on their Debian system. Sorry about that. While I try to make my columns non-release specific, I sometimes slip into the “I'm running this here system” trap. To get the scoop on rc.local, I need to give you the scoop on run levels and a handful of scripts that run each time your system comes up. What gets executed at bootup is partly defined by symbolic links located in an rc#.d file. The “#” represents a number corresponding to the run level.
What is this run level? In your /etc/inittab file, you will find an entry that says something like this.
Note the “3” at the beginning of the line. This tells me that when the system comes up, it will, by default, switch to run level 3 (which is full multiuser mode with a command line login). If your system says “5”, this tells me you are booting directly to the graphical desktop. What starts at each of these run levels will be found in an accompanying /etc/rc#.d directory, in my case, rc3.d. Yeah, it's true. I'm still a command line guy who starts his X desktop with the startx command.
Anyhow, back to this rc.local file. On a Red Hat (or Mandrake) system, you'll find it hiding under /etc/init.d. I hear someone in the back yelling, “Wait a minute and stop right there! Didn't you just say that it would be in the rc3.d directory?” It is. More or less.
If you change directory to /etc/rc3.d , you'll see a number of script files either starting with an “S” or a “K”. Do an ls -l and you will notice they are all symbolic links pointing back to a directory somewhere else. On a SuSE system, the rc#.d directories are under /sbin/init.d, but you will still find those “S” or “K” files and they point to /sbin/init.d. In the case of my Red Hat system, they point back to the /etc/rc.d/init.d directory.
lrwxrwxrwx 1 root root 11 Jul 12 16:09 /etc/rc.d/rc5.d/S99local -> ../rc.local
On a Debian system, these scripts point back to /etc/init.d which is where I would create my rc.local file. On my own system, it turns out that rc.local is executed by a call to S99local. On a Debian system, for instance, look for (or create) an S99local file under the appropriate run level directory. My use (or Red Hat's) of S99local is (to some degree) convention, but you could, if you wanted to, be somewhat more arbitrary. The first part of that name, the “S”, means “start” (a “K” means “kill”) and the 99 is simply a high enough number that it is likely the last thing your system executes on boot. The “local” part is just a name that means something to me. You might call it “rclocal” or “systemlocal” or “iceberg”. So, if I want this file started with my run level 3 on that Debian system, I would create a symbolic link like this.
ln -s /etc/init.d/rc.local /etc/rc3.d/S99local
Make sure (of course) that the script is executable. Now, let's get back to some of those tweaks.
What I gave you last time were all network tweaks. This time around, I want to show you a few file system tricks. In past lives (working with other UNIXes), the systems I administered ran complex databases, often with hundreds of users. I'm fond of the following tweaks because they represent parameters that require a kernel rebuild if you find yourself starting to run low. You made your best guess, but invariably, it would be kernel rebuild time soon enough. With Linux, these parameters are simple /proc tweaks. If you are running a busy database system with a large number of users, this is one you might run into. The “file-max” parameter defines the maximum number of open files on your system at any given time. For most, the default “4096” is plenty. For a busier system, you might want to push that limit up somewhat. As an example, let's double that number.
echo "8192" > /proc/sys/fs/file-max
If you get errors stating that you are running out of file handles, it's definitely time to change that number, but don't wait for the users to start ringing. Without waiting for errors, you can take a look under the hood and see when this limit is approaching. (Preventative maintenance. What a concept.) If you do a cat on /proc/sys/fs/file-nr, you will get three numbers. The third will be your file-max. The first and second are the number of allocated file handles and the number of actual used file handles. Why the two numbers? When the Linux kernel allocates a file handle, it does not release it. If you do increase the file-max value, then you should also increase inode-max as well. Considering that each open file requires an inode for stdin, stdout (and possibly, a network socket) this needs to be somewhat higher than your file-max. Take your file-max value, triple it and write it back out to inode-max.
echo "24576" > /proc/sys/fs/inode-max
Busy web server? News server? Here's another tweak for your files, and this one has nothing to do with /proc. One of the options for the mount command is “noatime”. In other words, do not (don't you even think about it) update the access time on visited files. Each time a file is read, the access time is updated which can yield useful information about file usage (with the find command, for instance). Odds are, you may not need that information. In the case of a web server getting a few thousand hits a day (an hour?), this little change can make a difference. Historically, this option was a suggestion for directories on news servers. Today, we are usually talking web servers. This is an environment where small files are accessed over and over again ( vs. a database environment which traditionally has comparatively few, large files).
To mount a filesystem noatime, use the “-o” flag, like this. In this example, we'll use the pretend drive “hda5”.
mount -o noatime /dev/hda5 /data1
If you wanted this to happen automatically, you could also edit your /etc/fstab file so that you have an entry similar to this one.
/dev/hda5 /data1 ext2 defaults,noatime 1 2
Enough file tweaking for one day. Before we wrap up, I want to talk about the need for tweaking. Another way to look at it is this: how do you know that you might be hitting a kind of wall? One of the surest ways is simply to monitor your system's performance through the various tools your system already has. The most basic of these is a little program called uptime which most of us use to drive our Windows-using friends crazy. “Ah, I see that you've rebooted twice today already. Let me run uptime and see what I get.
1:21pm up 127 days, 6:02, 4 users, load average: 0.31, 0.29, 0.26
”My, my, my. Would you look at that? 127 days, 6 hours and 2 minutes without a reboot.“
Before we get into too much trouble, let's see what else the program tells you. There are four users logged in. The load average is 0.31 over the last minute, 0.29 over the last 5 minutes and 0.26 over the last 15 minutes. Load average indicates roughly the number of processes in the CPU's run queue; that is, the number of processes active or waiting to execute. If it helps any, you might think of it as the number of patients in the waiting room to see the doctor. In this case, I had an average of one third of one process waiting to be dealt with. The higher the number for load average, the more likely your system is starting to suffer under an excessive load. As the saying goes, your mileage may vary, but I tend to think of anything under four as acceptable. Any higher and it starts feeling slow. I've seen systems running around 15 to 20 and let me tell you, it's ugly.
If those numbers are high, the very next question is ”Why?". Other questions follow. What is it that is holding things up? If I am running out of something, how do I know what that something is? And those are just the types of questions that I want to consider when next we meet here at the corner. Until then, give Tux a tweak. You both might enjoy it.
Practical Task Scheduling Deployment
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.View Now!
|The Firebird Project's Firebird Relational Database||Jul 29, 2016|
|Stunnel Security for Oracle||Jul 28, 2016|
|SUSE LLC's SUSE Manager||Jul 21, 2016|
|My +1 Sword of Productivity||Jul 20, 2016|
|Non-Linux FOSS: Caffeine!||Jul 19, 2016|
|Murat Yener and Onur Dundar's Expert Android Studio (Wrox)||Jul 18, 2016|
- The Firebird Project's Firebird Relational Database
- Stunnel Security for Oracle
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- SUSE LLC's SUSE Manager
- Managing Linux Using Puppet
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- Parsing an RSS News Feed with a Bash Script
- Google's SwiftShader Released
- Doing for User Space What We Did for Kernel Space
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide