More on Using Bash's Built-in /dev/tcp File (TCP/IP)

 in

If you saw yesterday's Tech Tip and were looking for more on using TCP/IP with bash's built-in /dev/tcp device file then read on. Here, we'll both read from, and write to a socket.

Before I go any further, let me state that this is based on something I discovered here on Dave Smith's Blog. All I've done here is added a few improvements based on the comments to the original post. I've also added a bit of additional explanation.

The following script fetches the front page from Google:

exec 3<>/dev/tcp/www.google.com/80
echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
cat <&3

Pretty simple, just 3 lines. The first line may be a bit confusing if you haven't seen this type of thing before. This line causes file descriptor 3 to be opened for reading and writing on the specified TCP/IP socket. This is a special form of the exec statement. From the bash man page:

exec [-cl] [-a name] [command [arguments]]

... If command is not specified, any redirections take effect in the current shell, and the return status is 0.

So using exec without a command is a way to open files in the current shell.

After the socket is open we send our HTTP request out the socket with the echo ... >&3 command. The request consists of:

GET / HTTP/1.1
host: http://www.google.com
Connection: close

Each line is followed by a carriage-return and newline, and all the headers are followed by a blank line to signal the end of the request (this is all standard HTTP stuff).

Next we read the response out of the socket using cat <&3, which reads the response and prints it out. The response being the main HTML page from Google:

$ bash tcp.sh
HTTP/1.1 200 OK
Date: Wed, 30 Sep 2009 17:28:36 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
Set-Cookie: PREF=ID=...
Set-Cookie: NID=27=...
Server: gws
X-XSS-Protection: 0
Transfer-Encoding: chunked
Connection: close

fef
<!doctype html><html><head><meta ...

And that's it, with just a few more lines of code you could have your own bash based browser... well maybe not.

______________________

Mitch Frazier is an Associate Editor for Linux Journal.

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

By the time you're done improving the script....

ChronoFish's picture

You'll have re-built "wget". If you need the raw headers, a telnet script will work and have the added benefit of being shell independent... But cool none the less.

-CF

Host header

Anonymous's picture

Great Article, but I'm going to point out something nitpicky:

The host header "host: http://www.google.com" should be changed to "Host: www.google.com" to be HTTP 1.1 compliant. Thanks!

Newlines and shebang

Benjie Gillam's picture

Technically you should either do "echo -en" or remove the last "\n" from the echo string, otherwise what is actually sent to Google is "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n\n" with two "\n"s at the end. This is of particular importance for POST requests (or other requests with a payload).

I'd also suggest you add the shebang #!/bin/bash to the top of the script - I think at least Ubuntu generally uses dash instead of bash for /bin/sh which may cause the "file" to appear missing even if supported by bash.

What version of bash brings /dev/tcp support? Neither Ubuntu Hardy (64bit) nor Ubuntu Jaunty (32bit) seem to support it out of the box. Unless it requires installing an additional package?

Echo

Mitch Frazier's picture

Adding the -n to the echo command is a good idea. Removing the last newline would work also, though I think that would "look" confusing since there'd be a carriage return "hanging" out there in the middle of nowhere, so to speak.

Not sure which version of bash included this, but it's a compile time option and ubuntu (and debian) don't enable it. See the comments on the original tech tip.

Mitch Frazier is an Associate Editor for Linux Journal.

This doesn't work: exec

Anonymous's picture

This doesn't work:

exec 3<>/dev/tcp/www.google.com/80
bash: /dev/tcp/www.google.com/80: No such file or directory

bash --version
GNU bash, version 3.2.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2007 Free Software Foundation, Inc.

See Tech Tip Comments

Mitch Frazier's picture

This means your version of bash wasn't compiled with /dev/tcp support. See the comments attached to the original Tech Tip.

Mitch Frazier is an Associate Editor for Linux Journal.

what about closing the socket

piavlo's picture

you forgot to mention how to close the socket to avoid the CLOSE_WAIT :) - which would be "exec 3>&-"

shell> exec 3<>/dev/tcp/www.google.com/80
shell> netstat -anpt | grep 80 | grep bash
tcp 0 0 1.2.3.4:58463 209.85.129.99:80 ESTABLISHED7812/-bash
shell> echo -e "GET / HTTP/1.1\r\nhost: http://www.google.com\r\nConnection: close\r\n\r\n" >&3
shell> netstat -anpt | grep 80 | grep bash
tcp 833 0 1.2.3.4:58463 209.85.129.99:80 CLOSE_WAIT 7812/-bash
shell> cat <&3
HTTP/1.1 302 Found
Location: http://www.google.co.il/
Cache-Control: private
Content-Type: text/html; charset=UTF-8
Set-Cookie: PREF=ID=f8d725d8e4255cbd:TM=1254384103:LM=1254384103:S=U8mS08Olic23lpHx; expires=Sat, 01-Oct-2011 08:01:43 GMT; path=/; domain=.google.com
Set-Cookie: NID=27=BPe4nHbiomJwYiJ6f0YXwVcKrv9ffW8VcrnJJ_bNNWaWyH6nn6gGE1lh7nAUxEswSmFf9d59lX8a-3EbHf9_YrxhqCd9IBGF6hZjeKHtHtfG97be79Bq3mvf4tq8vfAY; expires=Fri, 02-Apr-2010 08:01:43 GMT; path=/; domain=.google.com; HttpOnly
Date: Thu, 01 Oct 2009 08:01:43 GMT
Server: gws
Content-Length: 221
X-XSS-Protection: 0
Connection: close

302 Moved
302 Moved
The document has moved
here.

shell> netstat -anpt | grep 80 | grep bash
tcp 0 0 1.2.3.4:58463 209.85.129.99:80 CLOSE_WAIT 7812/-bash
shell> exec 3>&-
shell> netstat -anpt | grep 80 | grep bash
shell>

Close Wait

Mitch Frazier's picture

Another good addition.

Mitch Frazier is an Associate Editor for Linux Journal.

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix