Usenix Tutorials Answer Deep Questions

As a Linux administrator, you have a lot
of choices about where to go for training. For a good mixture of
deep and practical knowledge that doesn't duplicate what you can
get from books, man pages and experimentation, it's hard to beat
the tutorial section of the Usenix Technical Conference.I attended three tutorials: "Inside the Linux Kernel",
conducted by Ted Ts'o and intended for people considering getting
into kernel development; "Topics in UNIX and Linux Administration",
a selection of new items system administrators should be concerned
about and conducted by gurus Trent Hein, Ned McClain and Evi
Nemeth; and "Exploring the Potential of LDAP", taught by Samba
contributor Gerald Carter.Ts'o said the kernel tutorial was "enough background material
that you'll be able to dive into the sources", and he did give a
useful road map for people exploring the kernel without slowing
down and going over actual source code. He made the useful
assumption that if he gives you a map of how things work, you'll be
able to figure out the details when you get there.He did cover some items of historic interest from the 2.2 and
previous kernels, but mainly he covered 2.4 with an occasional peek
at or speculation about what's happening in 2.5. Instead of simply
presenting how things work, Ts'o gave the audience many of the
performance considerations behind the design decisions. For
example, he covered the older interrupt bottom halves and the new
(with 2.4) tasklets in the context of the
free
performance testing that Mindcraft thoughtfully gave the 2.2.6
kernel in 1999. Only one bottom half can be run on any CPU,
while different CPUs can run different tasklets.How effective was the tasklet innovation? The fact that
Mindcraft's home
page still links to the 2.2.6 tests should tell you
something. Performance wars aside, a long-time kernel contributor's
view of this anecdote helps wannabe kernel hackers see what's
important to the kernel team.Other considerations when writing kernel code include the
fact that every process has its own 8Kb kernel stack, which is not
enough to do serious recursion or big arrays on the stack. Also,
interrupts can borrow space on the kernel stack of the currently
running process.Finally, although the session was mainly an overview for
kernel developers, Ts'o also laid out some good tips for people
using an unmodified kernel. For example, the old advice about
having a swap partition equal to at least the size of physical RAM
is now obsolete. 2.4 and later kernels will discard, without
hitting disk, any page already present on the swap partition, so it
is better to have a swap partition twice the size of physical RAM
to increase the likelihood of this happening.Adding more RAM still is your best performance bargain. In
the "advantages of working for IBM" department, Ts'o's laptop is
equipped with 512MB, and he mentioned that the IBM ThinkPad T23
will accept up to 1GB. A final performance tip is that when making
a new ext2 or ext3 filesystem, you ordinarily should make the block
size the same as the kernel page size.All in all, this was a worthwhile tutorial for people
starting kernel development or doing serious performance tuning and
interested in developing an accurate mental model of how things
work.Technology and Policies for System
AdministrationIn "Topics in UNIX and Linux Administration, Part I", Trent
Hein, Ned McClain and Evi Nemeth, three of the coauthors of
Linux
Administration Handbook, covered some new topics
that aren't in any book yet, but that professional system
administrators will need to learn about.Most relevant in today's world of bigger and bigger Linux
servers, with bigger and bigger responsibilities, was the Linux
Logical Volume Manager section. With LVM, you can grow and shrink
filesystems without repartitioning the disk and, in the case of
ext3 or ReiserFS, without even remounting the filesystem. Another
advantage of LVM, perhaps more relevant for day-to-day-use, is the
ability to snapshot a filesystem--to create quickly what looks like
a complete copy, frozen in time.The snapshot doesn't actually contain a whole copy of all the
data from the original, of course. It only stores unchanged
versions of any blocks that get changed in the original. At the
time you create the snapshot, you specify how much space to allow.
As you might guess, in most cases you should take the snapshot,
dump it to tape and then remove it.While the LVM portion of the tutorial was Linux-specific, the
rest was relevant to almost any OS. BIND 9 is a total rewrite and
includes some features you can start using now and others for which
some important decisions still need to be made. You can start using
TSIG/TKEY to authenticate zone transfers among your name servers
now, because you create and manage the necessary keys, Nemeth said.
How the TLDs, such as .com, will handle key management for the more
complex DNSSEC is still being worked out.No section on security would be complete without covering the
human element. In the matter of making users choose good passwords,
it's better to educate them than simply to reject easy-to-guess
passwords. Make sure that your passwd utility gives a reason, such
as "too short" or "based on a dictionary word", that helps the user
pick a good password.Educating users about security really helps. "You need to get
your users to buy in a little", Nemeth said, and educating them
about security helps this happen. Also, security measures can end
up reducing security when users work around them. "I hate password
aging programs", Nemeth said. They make users rotate among two or
three passwords--however few they can get away with--and keep the
password aging tool happy.Among the nastiest of the system administrator's legal
problems is discovering unauthorized copies of proprietary
software. In many cases, your immediate manager may be among the
offenders and might tell you to leave it alone. But this is not an
option. "Sometimes you have to go further up the management chain
and make your complaint in writing. Sometimes you have to quit",
Nemeth said.Hiring and firing is another difficult problem. Nemeth
recommends that you give a questionnaire to job applicants and add
a few bogus technologies to measure the applicant's "bullshit
level". If someone says he or she can "wire up an MX connector"
without supervision, be wary. When checking references, remember
that some of them won't be willing to say explicitly anything
negative. "Listen really carefully to the previous boss", she
said.In a large company or the public sector, it may be
significantly easier to let someone go in the first few months of
employment, so don't delay. In any case, "You'll lose good people
if you don't deal with the folks who are not doing their
job."One final piece of common sense is that when you have several
administrators working on the same group of machines, keep a
"diary" for each one that covers hardware and software changes and
any unusual events. And label everything.From Buzzword to Your Next ProjectGerald Carter's LDAP tutorial addressed what may be among the
hardest problems system administrators will face some time soon:
setting up a secure, sustainable, company-wide directory to
integrate with many vendors' server programs. Everything seems to
do LDAP: login, Samba, DNS, web servers, IMAP and POP servers, and
more. Carter polled the audience about what LDAP servers they are
deploying, and the ones from iPlanet, formerly Netscape, and from
the free OpenLDAP project seem to be among the most common.With the sheer number of programs that want to talk LDAP,
there could not possibly have been enough time to cover too much
detail about each, so the tutorial concentrated on high-level
issues of schema design considerations, security and answering
questions from the audience. Integration with PAM got detailed
coverage, and PAM is the ticket to make LDAP-based authentication
work as a simple replacement for NIS password files.Carter is finishing up a book on LDAP for O'Reilly and
Associates, and this tutorial couldn't help but make me think I
might need it. Like the other tutorials, this one didn't insult the
audience's intelligence or man-page-finding skills.Thanks to the depth of experience of all the instructors, all
three Usenix tutorials made the abundant Q&A time worthwhile.
When a computer instructor says, "That's a good question", you
might expect the next words out of his or her mouth to be, "I'll
look it up and mail you." That's not the case here.Also absent were rathole discussions, time wasting
schedule-tweaking, administrivia and A/V related education-suckage.
Usenix has been doing tutorials for a long time, and their
professionalism shows. Breaks were well-timed for catching up on
notes, mail checking, coffee drinking and talking with other
attendees and instructors.The handout materials and slides were thin but valuable. If
you work in an environment where people consider you the Linux or
UNIX expert, you can count on maximizing ROI on your training
budget by attending the information-dense Usenix tutorials.Don Marti is technical
editor of Linux Journal.

email: dmarti@ssc.com