Tips for Optimizing Linux Memory Usage

In a previous issue, Jeff discussed ways to reduce disk space usage under Linux. In this sequel article, he shows some useful techniques for making the best use of another valuable resource—memory.
The X Window System

If you ran the command line described earlier, one of your largest binaries was probably the X server. The X Window system takes a lot of memory resources.

The first question to consider is, do you really need to run X? Using the virtual consoles and selection service you can have multiple windows supporting cut & paste of text using a mouse. Particularly while performing large compiles (such as the kernel), you should consider the option of simply not running X.

There is also a windowing system called “mgr” than can be used as an alternative to X, but requires less memory.

If you decide to use X, then you can obtain replacements for some of the standard tools that require less resources. “Rxvt” is similar to xterm, but requires significantly less memory. The window manager “fvwm” will also use less resources than others, and “rclock” is a small X-based clock program. These three tools, written by Robert Nation, can make running X feasible on a machine that constantly swapped before.

How many programs do you run on the X desktop? Run “top” to see how much memory is being taken by xclock, xeyes, xload, and all those other goodies you think you need.

The “Tiny X” package, put together by Craig I. Hagan, contains the Korn shell, fvwm window manager, rxvt, rclock, X server, and the minimum of other files needed to run X. The package is small enough to fit on one 3.5" floppy disk. Also included are some useful notes on saving memory under X.

With the techniques described here, you can run small X applications reasonably well on a machine with only 4 megabytes of memory. On machines with more memory, the same methods will allow you to run larger applications and free up memory to use for disk buffering.

Conclusions

By combining the techniques I've described, the net effect on system performance can be well worth the effort. I encourage you to experiment, and along the way you'll almost certainly learn something new.

For More Information

The software mentioned in this article is available on a number of Internet archive sites, including sunsite.unc.edu and tsx-11.mit.edu. I suggest getting a copy of the Linux Software Map to help track down the software you need.

If you want to learn more about how the Linux kernel implements memory management, check out ”The Linux Kernel Hackers' Guide“, by Michael K. Johnson, part of the Linux documentation project. Appendix A of that document includes an extensive bibliography of books covering operating system concepts in general.

”How to Maximize the Performance of X" is periodically posted to the Usenet newsgroup news.answers, and contains more ideas for improving X performance on small systems.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Buffer+Cache grows with time and is not freed when nedded

Vikash's picture

This is wrt a telecom hardware that is supposed to support some no of calls/UEs. It has a Cavium Octeon processor and an SMP LINUX by WindRiver systems.
Now, when I admit a new call, the used grows (okay), after some 50 UEs are introduced, the system is left with some 40 K of memory and the buffer and cache part has grown so much.
TOTAL USED FREE Buf/Cache
15:28:40 IST 2010 Mem: 1383 702 680 0 0 141
15:45:04 IST 2010 Mem: 1383 965 417 0 0 328
16:06:31 IST 2010 Mem: 1383 1341 41 0 2 693

Isnt the LINUX kernel supposed to free the buffer/cached part when the need for memory needed by new application come up..?
Please someone help me here. Any other info needed, pls let me know.

-Vikash

Need to tune memory

Anonymous's picture

cache memory is getting full very often. server is getting into crashing state. Nothing written in the swap space.

need ideas to tune the memory.

RAM 64G
swap 2 G

...

JShuford's picture

Thank you...

...I'm not just a "troll", but also a subscriber!

nice article. It will be

Manish Madhukar's picture

nice article. It will be great if you also explain the difference between cache and buffers as well...

-bash-3.2$ free -m
total used free shared buffers cached
Mem: 15364 6738 8625 0 211 4011
-/+ buffers/cache: 2514 12849
Swap: 12001 0 12001

Regards
Manish

Its really nice to see the

Madhusudhanan's picture

Its really nice to see the article written in 1994 still helps us, but things have slightly changed a bit since then.

Regarding your query,
The first row, labeled Mem, displays physical memory utilization, including the amount of memory allocated to buffers and caches. A buffer, also called buffer memory, is usually defined as a portion of memory that is set aside as a temporary holding place for data that is being sent to or received from an external device, such as a HDD, keyboard, printer or network.

The second line of data, which begins with -/+ buffers/cache, shows the amount of physical memory currently devoted to system buffer cache. This is particularly meaningful with regard to application programs, as all data accessed from files on the system that are performed through the use of read() and write() system calls1 pass through this cache. This cache can greatly speed up access to data by reducing or eliminating the need to read from or write to the HDD or other disk.

The third row, which begins with Swap, shows the total swap space as well as how much of it is currently in use and how much is still available.

CREDIT: http://www.linfo.org/free.html

- then again...

omelette's picture

... turns out that it was the linux kernel itself that was to blame;

http://www.intellinuxwireless.org/bugzilla/show_bug.cgi?id=1716

let's be careful out there... :)

things are not so clear-cut...

omelette's picture

Nice article, unfortunately like the dozens of others in a similar vein one can google, they all imply that the 'seekers' lack of knowledge of what is discussed above is all that's at issue - imo, this is not (always!) the case, as some (many?) programs leak memory like a sieve! For instance, at this moment my Debian lenny KDE ProDuo 1Gig laptop, up for 6-8hrs but with all apps closed produces the following memory-footprint - again, there's NOTHING running apart form the desktop:

omelette@debian:~$ free
total used free shared buffers cached
Mem: 1034636 956996 77640 0 3944 153828
-/+ buffers/cache: 799224 235412
Swap: 1052216 84856 967360

One reboot later, here is the results:

omelette@debian:~$ free
total used free shared buffers cached
Mem: 1034636 167316 867320 0 10788 88568
-/+ buffers/cache: 67960 966676
Swap: 1052216 0 1052216

Some difference! - clearly, 400-500meg of memory is not being released by apps and linux seems completely oblivious to this! Note, running any decent-sized app in this out-of-memory state sees swap grow proportionally, so linux definitely believes it has run out of memory! And this is not down to Debian or a specific kernel either - I have just transferred over from Ubuntu which performs abysmally as well (far worse it appears, though this may be 'cos Gnome uses more resources than KDE, so it's more noticeable) Ironically, I tested Debian extensively for one day on an 'old' Athlon-based computer, where it performed magnificently, before sticking it on the laptop as well. I'm now starting to wonder might it have something to do with dual-processors... BTW, I'm not out to bash linux here - I have Ubuntu running continuously as a server of sorts on a really old 700mhz Celeron without problems. Anyway, as 'top' is just-about useless in resolving this, I've just installed 'valgrind' in the hope that it will somehow shed some light on the problem (if I can ever figure out how to use it...)

Very useful

Pravin's picture

Nice Article. I was wondering how free memory is so less on my system and large memory is shown in buffers? This article cleared it.

The total of free is always

jithu's picture

The total of free is always less that the amount of physical memory (RAM) actually present on the machine. Why is it ?

you have told The “total” memory is the amount available after loading the kernel. so is the difference the space that is occupied by kernel and its data structures like page table etc ?

Nice article but would be

Anonymous's picture

Nice article but would be good if you explained how to do some of these things - particularly how to stop processes from running at startup like reducing the number of getty's etc.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState