GAR: Automating Entire OS Builds
One way to think of GAR is as a consistent interface for compiling a piece of software. No matter what mechanism the program uses to compile or install, a make install will install the program. The user need not care whether a program is a Python script or a C program that uses Imake.
In fact, the system will even satisfy dependencies. If the robotfindskitten program uses libncurses, the GAR system will pause the robotfindskitten build process right before the configure stage and then go install libncurses.
The GAR system is also centrally configurable, allowing the user to specify installation directories, CFLAGS and other useful environment variables. It is possible to configure default download locations, so that you can grab the code from a local cache or a nearby mirror of all the files.
GAR was designed with the notion that the build system and the target system may be different machines. It is also possible that the build and target system are different architectures. The following changes to gar.conf.mk show a possible way to configure GAR for a cross-compiling environment:
prefix = /usr/local/build/mips-build LDFLAGS += -L$(libdir) -L/usr/local/lib-mips CC = /usr/local/bin/gcc-mipsel LD_LIBRARY_PATH += :$(libdir):/usr/local/lib-mips export prefix LDFLAGS CC LD_LIBRARY_PATH
All of these settings will propagate down into each individual package.
The best way to understand the process of making a new package is to look at an example. The following is the relevant parts of the grep package:
GARNAME = grep GARVERSION = 2.4.2 CATEGORIES = utils MASTER_SITES = ftp://ftp.gnu.org/pub/gnu/grep/ DISTFILES = $(DISTNAME).tar.gz CONFIGURE_SCRIPTS = $(WORKSRC)/configure BUILD_SCRIPTS = $(WORKSRC)/Makefile INSTALL_SCRIPTS = $(WORKSRC)/Makefile CONFIGURE_ARGS = $(DIRPATHS) include ../../gar.mk
The first few lines set some basic bookkeeping information. The GARNAME and GARVERSION refer to the package name and version string as the GAR system will manipulate them. They are also used to create the convenience variable DISTNAME, which is $(GARNAME)-$(GARVERSION) by default, since most GNU or automake-using packages name their tarballs in that fashion.
The actual build of the system depends on the CONFIGURE_SCRIPTS, BUILD_SCRIPTS and INSTALL_SCRIPTS variables. These point to a space-separated list of files that are essential to the configure, build and install steps (respectively).
The GAR system will know what to do with most types of scripts. It knows to run CONFIGURE_SCRIPTs named ``configure'', and to run make or make install in the directory where a Makefile lives. It also knows that Imakefiles use xmkmf, and so forth. For most packages, no extra work needs to be done here.
We usually need to specify the CONFIGURE_ARGS to include the directory settings that we define in gar.conf.mk. This currently requires setting it to include $(DIRPATHS).
Finally, we include the gar.mk library. This takes all of the variable setting we've done and puts it to good use. This allows your short Makefile to provide the seven basic targets described above. It must come last (or, at least, after all of the variables have been set) in order to function properly.
The automatic satisfaction of dependencies is deceptively simple. In order to specify a library dependency, for example, simply set the LIBDEPS variable to be a space-separated list of paths from the base GAR directory to the package you depend on. For example, the GNU parted program has the following:
LIBDEPS = lib/e2fsprogs-libs
Before the configure step is run, GAR will cd into ../../lib/e2fsprogs-libs/ and run make install.
Quite often, the default behavior isn't what one would hope. You can pass parameters to a package's configure script with CONFIGURE_ARGS, but what if the GAR system doesn't know about your configuration script type? What if the configuration steps aren't enough? What if they're flat-out wrong? What if the same is true for the fetch, extract or install rules?
Fortunately, the system provides for this in a number of ways. What follows are a few of the mechanisms that a package maintainer can use to override or enhance the default behaviors.
For each of the seven basic targets, there exists slots for per-package pre- and post- rules. That means that the package maintainer can specify work to be done immediately before or after a rule.
To define a rule in make, simply place it at the beginning of a line, followed by a colon. The shell commands it should execute follow, preceded by a tab on each line.
As an example, let's consider the util-linux package. It doesn't use a standard autoconf-style configure script, but it can be configured by setting a variable at the top of the MCONFIG file. Thus, the end of our utils/util-linux/Makefile looks like the following:
CONFIGURE_SCRIPTS = $(WORKSRC)/configure BUILD_SCRIPTS = $(WORKSRC)/Makefile INSTALL_SCRIPTS = $(WORKSRC)/Makefile CONFIGURE_ARGS = $(DIRPATHS) pre-configure: echo "DESTDIR=$(prefix)" > $(WORKSRC)/MCONFIG.NEW cat $(WORKSRC)/MCONFIG << $(WORKSRC)/MCONFIG.NEW mv $(WORKSRC)/MCONFIG.NEW $(WORKSRC)/MCONFIG $(MAKECOOKIE) include ../../gar.mk
So before the configure script is run, the package-defined preconfigure rule adds code setting DESTDIR to the $(prefix) variable in MCONFIG.
The $(MAKECOOKIE) variable is a macro that creates a cookie file signifying the completion of the preconfigure step; $(MAKECOOKIE) performs some housekeeping that ensures rules are run only once, so it's important to end each rule with $(MAKECOOKIE).
As another example, the Bourne Again SHell (bash) can be linked to with the name ``sh'' in order to cause it to behave (somewhat) like a POSIX Bourne Shell. To do this, the end of our shells/bash/Makefile looks like:
CONFIGURE_SCRIPTS = $(WORKSRC)/configure BUILD_SCRIPTS = $(WORKSRC)/Makefile INSTALL_SCRIPTS = $(WORKSRC)/Makefile CONFIGURE_ARGS = $(DIRPATHS) post-install: (cd $(bindir); ln -sf bash sh) $(MAKECOOKIE) include ../../gar.mk
Thus creating the symbolic link to sh.
Practical Task Scheduling Deployment
July 20, 2016 12:00 pm CDT
One of the best things about the UNIX environment (aside from being stable and efficient) is the vast array of software tools available to help you do your job. Traditionally, a UNIX tool does only one thing, but does that one thing very well. For example, grep is very easy to use and can search vast amounts of data quickly. The find tool can find a particular file or files based on all kinds of criteria. It's pretty easy to string these tools together to build even more powerful tools, such as a tool that finds all of the .log files in the /home directory and searches each one for a particular entry. This erector-set mentality allows UNIX system administrators to seem to always have the right tool for the job.
Cron traditionally has been considered another such a tool for job scheduling, but is it enough? This webinar considers that very question. The first part builds on a previous Geek Guide, Beyond Cron, and briefly describes how to know when it might be time to consider upgrading your job scheduling infrastructure. The second part presents an actual planning and implementation framework.
Join Linux Journal's Mike Diehl and Pat Cameron of Help Systems.
Free to Linux Journal readers.Register Now!
- SUSE LLC's SUSE Manager
- Murat Yener and Onur Dundar's Expert Android Studio (Wrox)
- My +1 Sword of Productivity
- Non-Linux FOSS: Caffeine!
- Managing Linux Using Puppet
- Doing for User Space What We Did for Kernel Space
- SuperTuxKart 0.9.2 Released
- Parsing an RSS News Feed with a Bash Script
- Google's SwiftShader Released
- Rogue Wave Software's Zend Server
With all the industry talk about the benefits of Linux on Power and all the performance advantages offered by its open architecture, you may be considering a move in that direction. If you are thinking about analytics, big data and cloud computing, you would be right to evaluate Power. The idea of using commodity x86 hardware and replacing it every three years is an outdated cost model. It doesn’t consider the total cost of ownership, and it doesn’t consider the advantage of real processing power, high-availability and multithreading like a demon.
This ebook takes a look at some of the practical applications of the Linux on Power platform and ways you might bring all the performance power of this open architecture to bear for your organization. There are no smoke and mirrors here—just hard, cold, empirical evidence provided by independent sources. I also consider some innovative ways Linux on Power will be used in the future.Get the Guide