Gnu Queue: Linux Clustering Made Easy
All of this sounds great, you say. How do I obtain and install GNU Queue? You can download the latest release of GNU Queue from its web site at http://www.gnuqueue.org/. It's a participating project on SourceForge, and you can find all sorts of discussion forums, support forums and bug-tracking databases. Download the program from the web site, unpack it and then run:
./configure<\n> make install
from the top-level directory. Run make install and fire up the dæmon with queued -D & on each machine in your cluster.
For a quick reference on using the queue command to farm jobs out to the network, visit the GNU Queue home page. That's all there is to it!
Before installing GNU Queue on your cluster, you have to make a decision that is basically guided by whether you have root (administrative) privileges on your cluster. If you do, you'll probably want to install GNU Queue in a manner that makes it available to all the users on your site. This is the --enable-root option. On the other hand, if you're just an ordinary Jane or Joe on your cluster or want to see what the fuss is all about without giving away privileges, you can install GNU Queue as an ordinary user, the default mode of installation.
Yes, ordinary users can install GNU Queue as a batch processing system on your cluster! But, if another user wants to run GNU Queue, he'll have to change the port numbers in the source code to insure no one else is running GNU Queue. That's why it's better to let the system administrator install GNU Queue (with --enable-root option to the configure script) if you expect a lot of users will want to run GNU Queue on your cluster.
Once you've downloaded GNU Queue off the Net, the first thing to do is to unpack it using the tar command. Under Linux, this is just tar xzf filename, where file name is the name of the file (compressed with gzip and having either the .tar.gz or .tgz file extensions. On other systems it's a little bit more involved, since the tar installed by default is not GNU tar and doesn't support the zdecompression option. You'll need to explicitly run the gunzip decompression program: gunzip filename.tar.gz; tar xf filename.tar, where filename.tar.gz is the file, with .tar.gz extension, that you obtained from the network. (Savvy users might want to use the zcat filename.tar.gz|tar tf - trick, but this assumes the zcat program installed on your system can handle GNU zipped file. gunzip is part of the GNU gzip package; you can obtain it from ftp://ftp.gnu.org/.
So you've unpacked the distribution and you're sitting in the distribution's top-level directory. Now what? Well, if you're an ordinary Jane or Joe you install the program into the distribution directory by running ./configure followed by make install on each machine in your cluster. Then, fire up the dæmon with queued -D & on each machine in your cluster. If you want more details (or you're a system administrator), continue reading.
Run ./configure. If you're installing it on a system where you're not a superuser but an ordinary peon, configure sets the makefile to install GNU Queue into the current directory. queue will go into ./bin; queued dæmon will go into ./sbin; ./com/queue will be the shared spool directory; the host access control list file will go into ./share; and the queued pid files will go into ./var . If you want things to go somewhere else, run ./configure --prefix=dir, where dir is the top-level directory where you want things to be installed.
The default ./configure option is to install GNU Queue in the local directory for use by a single user only. System administrators should run the command ./configure --enable-root instead. When installing with the --enable-root option, configure sets the makefile to install GNU Queue under the /usr/local prefix. queue will go in /usr/local/bin; queued dæmon will go into /usr/local/sbin; /usr/local/com/queue will be the shared spool directory; the host access control list file will go into /usr/local/share; and the queued pid files will go into /usr/local/var. If you want things to go somewhere else, run the following:
./configure --enable-root<\n> --prefix=dir
where dir is the top-level directory where you want things to be installed.
./configure takes a number of additional options that you may wish to be aware of, including options for changing the paths of the various directories. ./configure --help gives a full listing of them. Here are a few examples, --bindir specifies where queue goes; --sbindir specifies where queued goes; --localstatedir states where the spool directory and queued pid files go; and --datadir lists where the host access control file goes. If ./configure fails inelegantly, make sure lex is installed. GNU flex is an implementation of lex available from the FSF, http://www.gnu.org/.
Now, run make to compile the programs. If your make complains about a syntax error in the Makefile, you'll need to run GNU Make which is hopefully already installed on your machine (perhaps as gmake or gnumake), but, if not, you can obtain it from the FSF at http://www.gnu.org/.
If all goes well, make install will install the programs into the directory you specified with ./configure. Missing directories will be created. The host name of the node make install is being run on will be added to the host access control list if it is not already there.
Now, try running Queue. Start up ./queued -D & on the local machine. (If you did a make install on the node, the host name should already be in the host access control list file.)
|Using Salt Stack and Vagrant for Drupal Development||May 20, 2013|
|Making Linux and Android Get Along (It's Not as Hard as It Sounds)||May 16, 2013|
|Drupal Is a Framework: Why Everyone Needs to Understand This||May 15, 2013|
|Home, My Backup Data Center||May 13, 2013|
|Non-Linux FOSS: Seashore||May 10, 2013|
|Trying to Tame the Tablet||May 08, 2013|
- Making Linux and Android Get Along (It's Not as Hard as It Sounds)
- Using Salt Stack and Vagrant for Drupal Development
- New Products
- Validate an E-Mail Address with PHP, the Right Way
- Drupal Is a Framework: Why Everyone Needs to Understand This
- A Topic for Discussion - Open Source Feature-Richness?
- Home, My Backup Data Center
- New Products
- RSS Feeds
- Tech Tip: Really Simple HTTP Server with Python
54 min 27 sec ago
- Automatically updating Guest Additions
2 hours 3 min ago
- I like your topic on android
2 hours 49 min ago
- Reply to comment | Linux Journal
3 hours 10 min ago
- This is the easiest tutorial
9 hours 25 min ago
- Ahh, the Koolaid.
15 hours 3 min ago
- git-annex assistant
21 hours 3 min ago
- direct cable connection
21 hours 25 min ago
- Agreed on AirDroid. With my
21 hours 36 min ago
- I just learned this
21 hours 40 min ago
Enter to Win an Adafruit Prototyping Pi Plate Kit for Raspberry Pi
It's Raspberry Pi month at Linux Journal. Each week in May, Adafruit will be giving away a Pi-related prize to a lucky, randomly drawn LJ reader. Winners will be announced weekly.
Fill out the fields below to enter to win this week's prize-- a Prototyping Pi Plate Kit for Raspberry Pi.
Congratulations to our winners so far:
- 5-8-13, Pi Starter Pack: Jack Davis
- 5-15-13, Pi Model B 512MB RAM: Patrick Dunn
- Next winner announced on 5-21-13!
Free Webinar: Linux Backup and Recovery
Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.
In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.