Home › Forums › Linux Support › Programming ›

wget question

Jul 28, 2009  By lmcdougall
 in

I will like to get a list of the files in the directory of an ftp site.
can I use wget to do it?

if yes, the how?

any other method that I can easily implement in bash script is welcome also.

Thanks

‹ Need some help with the sed command php script --> MSSQL error. Why? See details below ›

You could also do this using

Anonymous's picture
Submitted by Anonymous (not verified) on Tue, 11/10/2009 - 09:52.

You could also do this using perl's Net::FTP module:

#!/usr/bin/perl
use strict;
use warnings;
use Net::FTP;

#Change your.ftp-host.com to your ftp domain
my $ftp = Net::FTP->new("your.ftp-host.com", Debug => 0)
        or die "Can not connect: $@";
#Change UserName and PassWord to your login information
$ftp->login("UserName",'PassWord')
        or die "Can not log in: ", $ftp->message;
#If you want to list your root directory you can comment this out
#Or change public_html to where you want to get a file list
$ftp->cwd("public_html");

my @list = $ftp->ls($ftp->pwd);
my $listing = "list.txt";

open(LISTING, ">$listing");

foreach my $line (@list) {
        print LISTING $line . "\n";
}

close(LISTING)

Use "ftp"

Mitch Frazier's picture
Submitted by Mitch Frazier on Wed, 07/29/2009 - 13:47.

Use the ftp command. For example, to get the items in /pub/lj/listings from the LJ  ftp server: 

ftp ftp://ftp.linuxjournal.com <<EOF
dir /pub/lj/listings filelist.tmp
y
EOF
sed -e 's/.* //' <filelist.tmp >filelist.txt

This will put the names into filelist.txt. Note that the sed command will produce incorrect results if any of the file names contain blanks. In that case you'll have to use a different method of extracting the file names from the raw results file (filelist.tmp).

Mitch Frazier is an Associate Editor for Linux Journal.

Webcast
More Resources
How to Build an Optimal Hadoop Cluster to Store and Maintain Unlimited Amounts of Data Using Microservers

Realizing the promise of Apache® Hadoop® requires the effective deployment of compute, memory, storage and networking to achieve optimal results. With its flexibility and multitude of options, it is easy to over or under provision the server infrastructure, resulting in poor performance and high TCO. Join us for an in depth, technical discussion with industry experts from leading Hadoop and server companies who will provide insights into the key considerations for designing and deploying an optimal Hadoop cluster.

Learn More

Sponsored by AMD

White Paper
More Resources
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState