What's GNU: Bash—The GNU Shell

 in
Conclusion of an article started last month. While originally written by Brian Fox of the Free Software Foundation, bash is now maintained by Chet Ramey. In this article, Chet explains the history of shells and then goes on to explore features specific to bash.
File System Views

Since Berkeley introduced symbolic links in 4.2 BSD, one of their most annoying properties has been the “warping” to a completely different area of the file system when using cd, and the resultant non-intuitive behavior of “cd ..”. The Unix kernel treats symbolic links physically. When the kernel is translating a pathname in which one component is a symbolic link, it replaces all or part of the pathname while processing the link. If the contents of the symbolic link begin with a slash, the kernel replaces the pathname entirely; if not, the link contents replace the current component. In either case, the symbolic link is visible. If the link value is an absolute pathname, the user finds himself in a completely different part of the file system.

bash provides a logical view of the file system. In this default mode, command and filename completion and builtin commands such as cd and pushd which change the current working directory transparently follow symbolic links as if they were directories. The $PWD variable, which holds the shell's idea of the current working directory, depends on the path used to reach the directory rather than its physical location in the local file system hierarchy. For example:

$ cd /usr/local/bin
$ echo $PWD
/usr/local/bin
$ pwd
/usr/local/bin
$ /bin/pwd
/net/share/sun4/local/bin
$ cd ..
$ pwd
/usr/local
$ /bin/pwd
/net/share/sun4/local

One problem with this, of course, arises when programs that do not understand the shell's logical notion of the file system interpret “..” differently. This generally happens when bash completes filenames containing “..” according to a logical hierarchy which does not correspond to their physical location. For users who find this troublesome, a corresponding physical view of the file system is available:

$ cd /usr/local/bin
$ pwd
/usr/local/bin
$ set -o physical
$ pwd
/net/share/sun4/local/bin
Internationalization

One of the most significant improvements in version 1.13 of bash was the change to “eight-bit cleanliness”. Previous versions used the eighth bit of characters to mark whether or not they were quoted when performing word expansions. While this did not affect the majority of users, most of whom used only seven-bit ASCII characters, some found it confining. Beginning with version 1.13, bash implemented a different quoting mechanism that did not alter the eighth bit of characters. This allowed bash to manipulate files with “odd” characters in their names, but did nothing to help users enter those names, so version 1.13 introduced changes to readline that made it eight-bit clean as well. Options exist that force readline to attach no special significance to characters with the eighth bit set (the default behavior is to convert these characters to meta-prefixed key sequences) and to output these characters without conversion to meta-prefixed sequences. These changes, along with the expansion of keymaps to a full eight bits, enable readline to work with most of the ISO-8859 family of character sets, used by many European countries.

POSIX Mode

Although bash is intended to be POSIX.2 conformant, there are areas in which the default behavior is not compatible with the standard. For users who wish to operate in a strict POSIX.2 environment, bash implements a POSIX mode. When this mode is active, bash modifies its default operation where it differs from POSIX.2 to match the standard. POSIX mode is entered when bash is started with the “-o posix” option or when set -o posix is executed. For compatibility with other GNU software that attempts to be POSIX.2 compliant, bash also enters POSIX mode if either of the variables $POSIX_PEDANTIC or $POSIXLY_CORRECT is set when bash is started or assigned a value during execution. When bash is started in POSIX mode, for example, it sources the file named by the value of $ENV rather than the “normal” startup files.

Future Plans

There are several features that will be introduced in the next version of bash, version 1.14, and a number under consideration for future releases. This section will briefly detail the new features planned for version 1.14 and describe features that may appear in later versions.

The new features available in bash-1.14 answer several of the most common requests for enhancements. Most notably, there is a mechanism for including non-visible character sequences in prompts, such as those which cause a terminal to print characters in different colors or in standout mode. There was nothing preventing the use of these sequences in earlier versions, but the readline redisplay algorithm assumed each character occupied physical screen space and would wrap lines prematurely.

Readline has a few new variables, several new bindable commands, and some additional emacs mode default key bindings. A new history search mode has been implemented: in this mode, readline searches the history for lines beginning with the characters between the beginning of the current line and the cursor. The existing readline incremental search commands no longer match identical lines more than once. Filename completion now expands variables in directory names. The history expansion facilities are now nearly completely csh-compatible: missing modifiers have been added and history substitution has been extended.

Several of the features described earlier as appearing in future releases, such as set -o posix and $POSIX_PEDANTIC, are present in version 1.14. There is a new shell variable, OSTYPE, to which bash assigns a value that identifies the version of Unix it's running on (great for putting architecture-specific binary directories into the $PATH). Two variables have been renamed: $HISTCONTROL replaces $history_control , and $HOSTFILE replaces $hostname_completion_file. In both cases, the old names are accepted for backward compatibility. The ksh select construct, which allows the generation of simple menus, has been implemented. New capabilities have been added to existing variables: $auto_resume can now take values of exact or substring, and $HISTCONTROL understands the value ignoreboth, which combines the two previously acceptable values. The dirs builtin has acquired options to print out specific members of the directory stack. The $nolinks variable, which forces a physical view of the file system, has been superseded by the -P option to the set builtin (equivalent to set -o physical); the variable is retained for backward compatibility. The version string contained in $BASH_VERSION now includes an indication of the patch level as well as the “build version”. Some little-used features have been removed: the bye synonym for exit and the $NO_PROMPT_VARS variable are gone. There is now an organized test suite that can be run as a regression test when building a new version of bash.

The documentation has been thoroughly overhauled: there is a new manual page on the readline library and the info file has been updated to reflect the current version. As always, as many bugs as possible have been fixed, although some surely remain.

There are a few features that I hope to include in later bash releases. Some are based on work already done in other shells.

In addition to simple variables, a future release of bash will include one-dimensional arrays, using the ksh implementation of arrays as a model. Additions to the ksh syntax, such as varname=( ... ) to assign a list of words directly to an array and a mechanism to allow the read builtin to read a list of values directly into an array, would be desirable. Given those extensions, the ksh

“set -A” syntax may not be worth supporting (the -A option assigns a list of values to an array, but is a rather peculiar special case).

Some shells include a means of programmable word completion, where the user specifies on a per-command basis how the arguments of the command are to be treated when completion is attempted: as filenames, hostnames, executable files, and so on. The other aspects of the current bash implementation could remain as-is; the existing heuristics would still be valid. Only when completing the arguments to a simple command would the programmable completion be in effect.

It would also be nice to give the user finer-grained control over which commands are saved onto the history list. One proposal is for a variable, tentatively named HISTIGNORE, which would contain a colon-separated list of commands. Lines beginning with these commands, after the restrictions of $HISTCONTROL have been applied, would not be placed onto the history list. The shell pattern-matching capabilities could also be available when specifying the contents of $HISTIGNORE.

One thing that newer shells such as wksh (also known as dtksh) provide is a command to dynamically load code implementing additional builtin commands into a running shell. This new builtin would take an object file or shared library implementing the “body” of the builtin (xxx_builtin() for those familiar with bash internals) and a structure containing the name of the new command, the function to call when the new builtin is invoked (presumably defined in the shared object specified as an argument), and the documentation to be printed by the help command (possibly present in the shared object as well). It would manage the details of extending the internal table of builtins.

A few other builtins would also be desirable: two are the POSIX.2 getconf command, which prints the values of system configuration variables defined by POSIX.2, and a disown builtin, which causes a shell running with job control active to “forget about” one or more background jobs in its internal jobs table. Using getconf, for example, a user could retrieve a value for $PATH guaranteed to find all of the POSIX standard utilities, or find out how long filenames may be in the file system containing a specified directory.

There are no implementation timetables for any of these features, nor are there concrete plans to include them. If anyone has comments on these proposals, feel free to send me electronic mail.

______________________

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState