Web Client Programming Using Perl
Many users of Linux are initially attracted to the platform for its powerful web server capabilities. Developing a web site on the Linux platform is a very satisfying experience. After the fun of designing and developing the site is over, it moves to production. Once in production, a web site typically has uptime requirements; to ensure these are met, the site must be monitored. I use a monitoring system that requests pages from a number of web sites, several times a day. If any server does not serve a web page, it is retried. If again there is no response, I am alerted via pager. Using this system, I can respond to an outage even before a user can report a problem. It takes only a little Perl and Linux to make this happen.
Linux and Perl were the obvious choices for a web server monitoring and alerting facility. Linux provides the stability and flexibility in the platform, and Perl, using the LWP bundle of modules (also known as libwww), provides an excellent means to work with HTTP requests.
I will explain how to compile/install the components needed, how to create a simple Perl script that will “HTTP ping” a server (HTTPping.pl), how to send a page to a pager using Perl, and how to glue it all together into an industrial-strength web site monitoring solution (Monitor.pl).
The wide availability of modules that extend Perl's functionality is one of the language's strong suits. In the case of fetching web pages, there are several modules that could be used. LWP is the clear choice, because it is a fantastic set of modules giving Perl powerful control over HTTP and HTML. Before you can begin exploring LWP and how you can use it to monitor a web site, you must install it.
The LWP modules depend on several other modules. These may not be installed on your system. If not, they must be downloaded and installed. Table 1 contains the exact version I installed, in the order I installed them. Except for SSLeay (more on that later), you can download all these packages from CPAN (Comprehensive Perl Archive Network). If you are new to Perl, it would be a good idea to browse around CPAN for a bit. It contains an enormous number of support modules for Perl that can save a lot of programming time and costs.
Once you have downloaded the tar.gz files, they must be expanded using the tar command:
tar -zxvpf MIME-Base64-2.11.tar.gz
This will uncompress and unpack the package. Do this for all archives. When finished, you will have a directory full of the original archives and a new folder for each archive. Each folder contains the module's installer; run the installer for each package. Again, using the MIME::Base64 module as the example, change (using cd) to the HTML-Parser-2.22 directory and type the following commands:
perl Makefile.PL make make test make installThis is the typical way of installing Perl modules. However, packages vary, and you should always read the README or INSTALL file and other documentation that comes with any module.
If you plan to HTTP ping servers that use SSL (secure sockets layer), you must compile and install SSLeay and Crypt-SSLeay. SSLeay is a set of programs that provides the cryptographic routines needed for SSL. Crypt-SSLeay is the Perl module that serves as a wrapper for SSLeay. I suggest you read the documentation that comes with SSLeay. It will help especially if you run into any problems when compiling.
Execute the following commands in the directory to which you uncompressed the SSLeay-0.6.6b.tar.gz archive to install SSLeay:
./Configure linux-elf make depend make make rehash make test make install
Three important things about SSLeay are:
Don't try to use the newer versions of SSLeay. They will not work with Crypt-SSleay and the LWP bundle. The last stable version is SSLeay-0.6.6b.tar.gz.
Second, SSLeay leaks memory. If you choose to use SSLeay and LWP to repeatedly HTTP ping a site that uses SSL, you will probably experience some problems with memory leaks. By calling HTTPping.pl from Monitor.pl, the script executes, pinging the site once, then terminates; this reduces the potential for a memory leak to accumulate, thereby causing your application to crash.
Since SSLeay uses RSA, you should read the SSLeay FAQ (see references) to determine if you can legally use SSLeay and if you need a license from RSA.
|Speed Up Your Web Site with Varnish||Jun 19, 2013|
|Non-Linux FOSS: libnotify, OS X Style||Jun 18, 2013|
|Containers—Not Virtual Machines—Are the Future Cloud||Jun 17, 2013|
|Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer||Jun 12, 2013|
|Weechat, Irssi's Little Brother||Jun 11, 2013|
|One Tail Just Isn't Enough||Jun 07, 2013|
- Speed Up Your Web Site with Varnish
- Containers—Not Virtual Machines—Are the Future Cloud
- Linux Systems Administrator
- Non-Linux FOSS: libnotify, OS X Style
- Lock-Free Multi-Producer Multi-Consumer Queue on Ring Buffer
- Senior Perl Developer
- Technical Support Rep
- UX Designer
- RSS Feeds
- Reply to comment | Linux Journal
3 hours 33 min ago
- Yeah, user namespaces are
4 hours 49 min ago
- Cari Uang
8 hours 21 min ago
- user namespaces
11 hours 14 min ago
11 hours 40 min ago
- One advantage with VMs
14 hours 9 min ago
- about info
14 hours 42 min ago
14 hours 43 min ago
14 hours 44 min ago
14 hours 46 min ago
Free Webinar: Hadoop
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers
Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.
Some of key questions to be discussed are:
- What is the “typical” Hadoop cluster and what should be installed on the different machine types?
- Why should you consider the typical workload patterns when making your hardware decisions?
- Are all microservers created equal for Hadoop deployments?
- How do I plan for expansion if I require more compute, memory, storage or networking?