More on Using Bash's Built-in /dev/tcp File (TCP/IP)

 in

If you saw yesterday's Tech Tip and were looking for more on using TCP/IP with bash's built-in /dev/tcp device file then read on. Here, we'll both read from, and write to a socket.

Before I go any further, let me state that this is based on something I discovered here on Dave Smith's Blog. All I've done here is added a few improvements based on the comments to the original post. I've also added a bit of additional explanation.

The following script fetches the front page from Google:

exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
cat <&3

Pretty simple, just 3 lines. The first line may be a bit confusing if you haven't seen this type of thing before. This line causes file descriptor 3 to be opened for reading and writing on the specified TCP/IP socket. This is a special form of the exec statement. From the bash man page:

exec [-cl] [-a name] [command [arguments]]

... If command is not specified, any redirections take effect in the current shell, and the return status is 0.

So using exec without a command is a way to open files in the current shell.

After the socket is open we send our HTTP request out the socket with the echo ... >&3 command. The request consists of:

GET / HTTP/1.1
host: http://www.google.com
Connection: close

Each line is followed by a carriage-return and newline, and all the headers are followed by a blank line to signal the end of the request (this is all standard HTTP stuff).

Next we read the response out of the socket using cat <&3, which reads the response and prints it out. The response being the main HTML page from Google:

$ bash tcp.sh
HTTP/1.1 200 OK
Date: Wed, 30 Sep 2009 17:28:36 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=...
Set-Cookie: NID=27=...
Server: gws
X-XSS-Protection: 0
Transfer-Encoding: chunked
Connection: close

fef
<!doctype html><html><head><meta ...

And that's it, with just a few more lines of code you could have your own bash based browser... well maybe not.

______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

By the time you're done improving the script....

ChronoFish's picture

You'll have re-built "wget". If you need the raw headers, a telnet script will work and have the added benefit of being shell independent... But cool none the less.

-CF

Host header

Anonymous's picture

Great Article, but I'm going to point out something nitpicky:

The host header "host: http://www.google.com" should be changed to "Host: www.google.com" to be HTTP 1.1 compliant. Thanks!

Newlines and shebang

Benjie Gillam's picture

Technically you should either do "echo -en" or remove the last "\n" from the echo string, otherwise what is actually sent to Google is "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n\n" with two "\n"s at the end. This is of particular importance for POST requests (or other requests with a payload).

I'd also suggest you add the shebang #!/bin/bash to the top of the script - I think at least Ubuntu generally uses dash instead of bash for /bin/sh which may cause the "file" to appear missing even if supported by bash.

What version of bash brings /dev/tcp support? Neither Ubuntu Hardy (64bit) nor Ubuntu Jaunty (32bit) seem to support it out of the box. Unless it requires installing an additional package?

Echo

Mitch Frazier's picture

Adding the -n to the echo command is a good idea. Removing the last newline would work also, though I think that would "look" confusing since there'd be a carriage return "hanging" out there in the middle of nowhere, so to speak.

Not sure which version of bash included this, but it's a compile time option and ubuntu (and debian) don't enable it. See the comments on the original tech tip.

Mitch Frazier is an Associate Editor for Linux Journal.

This doesn't work: exec

Anonymous's picture

This doesn't work:

exec 3<>/dev/tcp/www.google.com/80
bash: /dev/tcp/www.google.com/80: No such file or directory

bash --version
GNU bash, version 3.2.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.

See Tech Tip Comments

Mitch Frazier's picture

This means your version of bash wasn't compiled with /dev/tcp support. See the comments attached to the original Tech Tip.

Mitch Frazier is an Associate Editor for Linux Journal.

what about closing the socket

piavlo's picture

you forgot to mention how to close the socket to avoid the CLOSE_WAIT :) - which would be "exec 3>&-"

shell> exec 3<>/dev/tcp/www.google.com/80
shell> netstat -anpt | grep 80 | grep bash
tcp 0 0 1.2.3.4:58463 209.85.129.99:80 ESTABLISHED7812/-bash
shell> echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
shell> netstat -anpt | grep 80 | grep bash
tcp 833 0 1.2.3.4:58463 209.85.129.99:80 CLOSE_WAIT 7812/-bash
shell> cat <&3
HTTP/1.1 302 Found
Location: http://www.google.co.il/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=f8d725d8e4255cbd:TM=1254384103:LM=1254384103:S=U8mS08Olic23lpHx; expires=Sat, 01-Oct-2011 08:01:43 GMT; path=/; domain=.google.com
Set-Cookie: NID=27=BPe4nHbiomJwYiJ6f0YXwVcKrv9ffW8VcrnJJ_bNNWaWyH6nn6gGE1lh7nAUxEswSmFf9d59lX8a-3EbHf9_YrxhqCd9IBGF6hZjeKHtHtfG97be79Bq3mvf4tq8vfAY; expires=Fri, 02-Apr-2010 08:01:43 GMT; path=/; domain=.google.com; HttpOnly
Date: Thu, 01 Oct 2009 08:01:43 GMT
Server: gws
Content-Length: 221
X-XSS-Protection: 0
Connection: close

302 Moved
302 Moved
The document has moved
here.

shell> netstat -anpt | grep 80 | grep bash
tcp 0 0 1.2.3.4:58463 209.85.129.99:80 CLOSE_WAIT 7812/-bash
shell> exec 3>&-
shell> netstat -anpt | grep 80 | grep bash
shell>

Close Wait

Mitch Frazier's picture

Another good addition.

Mitch Frazier is an Associate Editor for Linux Journal.

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState