An Introduction to OpenSSL Programming, Part I of II

Do you have a burning need to build a simple web client and server pair? Here's why OpenSSL is for you.
Server

Our web server is mainly a mirror of the client but with a few twists. First, we fork() in order to let the server handle multiple clients. Second, we use OpenSSL's BIO APIs to read the client's request one line at a time, as well as to do buffered writes to the client. Finally, the server closure sequence is more complicated.

On Linux, the simplest way to write a server that can handle multiple clients is to create a new server process for each client that connects. We do that by calling fork() after accept() returns. Each new process executes independently and just exits when it's finished serving the client. Although this approach can be quite slow on busy web servers it's perfectly acceptable here. The main server accept loop is shown in Listing 5.

Listing 5. Server Accept Loop

After forking and creating the SSL object, the server calls SSL_accept(), which causes OpenSSL to perform the server side of the SSL handshake. As with SSL_connect(), because we're using blocking sockets, SSL_accept() will block until the entire handshake has completed. Thus, the only situation in which SSL_accept() will return is when the handshake has completed or an error has been detected. SSL_accept() returns 1 for success and 0 or negative for error.

OpenSSL's BIO objects are stackable to some extent. Thus, we can wrap an SSL object in a BIO (the ssl_bio object) and then wrap that BIO in a buffered BIO object, as shown below:

io=BIO_new(BIO_f_buffer());
ssl_bio=BIO_new(BIO_f_ssl());
BIO_set_ssl(ssl_bio,ssl,BIO_CLOSE);
BIO_push(io,ssl_bio);

This allows us to perform buffered reads and writes on the SSL connection by using the BIO_* functions on the new io object. At this point you might ask, why is this good? Primarily, it's a matter of programming convenience. It lets the programmer work in natural units (lines and characters) rather than SSL records.

Request

An HTTP request consists of a request line followed by a bunch of header lines and an optional body. The end of the header lines is indicated by a blank line (i.e., a pair of CRLFs, though sometimes broken clients will send a pair of LFs instead). The most convenient way to read the request line and headers is to read one line at a time until you see a blank line. We can do this using the OpenSSL_BIO_gets() call, as shown in Listing 6.

Listing 6. Reading the Request

The BIO_gets() call behaves analogously to the stdio fgets() call. It takes an arbitrary-sized buffer and a length and reads a line from the SSL connection into that buffer. The result is always null terminated (but includes the terminating LF). Thus, we simply read one line at a time until we get a line that consists of simply an LF or a CRLF.

Because we use a fixed-length buffer, it is possible, though unlikely, that we will get a line that's too long. In that event, the long line will be split over two lines. In the extremely unlikely event that the split happens right before the CRLF, the next line we read will consist of only the CRLF from the previous line. In this case we'll be fooled into thinking that the headers have finished prematurely. A real web server would check for this case, but it's not worth doing here. Note that no matter what the incoming line length is, there's no chance of a buffer overrun. All that can happen is that we'll misparse the headers.

Note that we really don't do anything with the HTTP request. We just read it and discard it. A real implementation would read the request line and the headers, figure out if there was a body and read that too.

The next step is to write the HTTP response and close the connection:

if((r=BIO_puts
  (io,"HTTP/1.0 200 OK\\r\\n"))<0)
  err_exit("Write error");
if((r=BIO_puts
  (io,"Server: EKRServer\\r\\n\\r\\n"))<0)
  err_exit("Write error");
if((r=BIO_puts
  (io,"Server test page\\r\\n"))<0)
  err_exit("Write error");
if((r=BIO_flush(io))<0)
  err_exit("Error flushing BIO");

Note that we're using BIO_puts() instead of SSL_write(). This allows us to write the response one line at a time but have the entire response written as a single SSL record. This is important because the cost of preparing an SSL record for transmission (computing the integrity check and encrypting it) is quite significant. Thus, it's a good idea to make the records as large as possible.

It's worth noting a couple of subtleties about using this kind of buffered write. First, you need to flush the buffer before you close. The SSL object has no knowledge that you've layered a BIO on top of it, so if you destroy the SSL connection you'll just leave the last chunk of data sitting in the buffer. The BIO_flush() call takes care of this. Also, by default, OpenSSL uses a 1024-byte buffer size for buffered BIOs. Because SSL records can be up to 16KB long, using a 1024-byte buffer can cause excessive fragmentation (and hence lower performance). You can use the BIO_ctrl() API to increase the buffer size.

Once we've finished transmitting the response, we need to send our close_notify. As before, this is done using SSL_shutdown(). Unfortunately, things get a bit trickier when the server closes first. Our first call to SSL_shutdown() sends the close_notify but doesn't look for it on the other side. Thus, it returns immediately but with a value of 0, indicating that the closure sequence isn't finished. It's then the application's responsibility to call SSL_shutdown() again.

It's possible to have two attitudes here. We could decide we've seen all of the HTTP request that we care about. We're not interested in anything else. Hence, we don't care whether the client sends a close_notify or not. Alternatively, we strictly obey the protocol and expect others to as well. Thus, we require a close_notify.

If we have the first attitude then life is simple. We call SSL_shutdown() to send our close_notify and then exit right away, regardless of whether the client has sent one or not. If we take the second attitude (which our sample server does), then life is more complicated because clients often don't behave correctly.

The first problem we face is that clients often don't send close_notifys all. In fact, some clients close the connection as soon as they've read the entire HTTP response (some versions of IE do this). When we send our close_notify, the other side may send a TCP RST segment, in which case the program will catch a SIGPIPE. We install a dummy SIGPIPE handler in initialize_ctx() to protect against this problem.

The second problem we face is that the client may not send a close_notify immediately in response to our close_notify. Some versions of Netscape require you to send a TCP FIN first. Thus, we call shutdown(s,1) before we call SSL_shutdown() the second time. When called with a “how” argument of 1, shutdown() sends a FIN but leaves the socket open for reading. The code to do the server shutdown is shown in Listing 7.

Listing 7. Calling SSL_shutdown()

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

hello Eric

Anonymous's picture

Thanks for the great article. You saved me a lot of work. The keys are out of date though. Can you drop a few lines as an appendix or here in comments on how to generate and self-sign new keys?

Best regards,
Reidar

how can check certificate against CRL in Linux using SSL command

laxman's picture

Hi,

How can client checks certificate against CRL(certification revocation List) using SSL libray functions.

I have loaded certifcate using X509_load_cert_crl_file(pLookup,"crl.pem",X509_FILETYPE_PEM);

but i am not getting how client can checks....

Thanks & Regards,
Laxman

Great article, need some info

saleem's picture

Hi,
Your article is simple and yet very informative.
Thanks for the article.

I am new to openssl, i am stuck with a problem. I need some info.

Due to some reasons the cert info is available in a buffer rather than
a file.
My client needs to load this and establish communication with the
server using openssl.

The buffer looks something like this....
char *gacacert =
"MIICLzCCAiswggGUoAMCAQICBgEYgSDT3DANBgkqhkiG9w0BAQUFADA0MRAwDgYD\n\
VQQKEwdlbnRydXN0MQwwCgYDVQQLEwNlbmcxEjAQBgNVBAMTCWdhTG9jYWxDQTAe\n\
Fw0wODAzMDUyMjQ3MzVaFw0yODAyMjkyMjQ3MzVaMDQxEDAOBgNVBAoTB2VudHJ1\n\
c3QxDDAKBgNVBAsTA2VuZzESMBAGA1UEAxMJZ2FMb2NhbENBMIGfMA0GCSqGSIb3\n\
DQEBAQUAA4GNADCBiQKBgQDW4ONrqPZ/Hc9Ft/vL1eD76XpbxhdmAezpjGK0aWa2\n\
2QCkDD6IpU3VxpW93+i8em2zgCV5fujbcJuNebk+Y24q3w8FVbba7BZGcaoatB99\n\
vdZ0gp/t/DXq9KsdxdlE2W/mKBCvxkkMsEnm5kHeHZXByouqPvIXGBsJORCH2ahB\n\
vwIDAQABo0gwRjASBgNVHRMBAf8ECDAGAQH/AgEAMBEGCWCGSAGG+EIBAQQEAwIA\n\
JDAdBgNVHQ4EFgQUIZVCc+92iSwt3CD3P9TYIJB6pLQwDQYJKoZIhvcNAQEFBQAD\n\
gYEAjZq3mZ/Q6F26BBd74Q5lJcABGTM4nB1mThaCJk//dLx6WhmWoXJoZD0//nYM\n\
UDvISCc4KtMZoe5qkO/BKJs9IwsXQyZiPl5bAtcfN6OmSe+fmNPMUKD1ck8l7WLu\n\
7k6hlBwrIIi05KhiYLY5i4ZbVh0+DyjIkXbv2GJj+g0CrEE=";

Could some one tell me how to load this buffer, communicate with the
server and perform the handshake?

Any code sample/example will be great.

Thanks,
Saleem

@ saleem Hi I am facing

Rakesh's picture

@ saleem

Hi

I am facing with the same issue.

I tried using this function to create a mem BIO which can be used to read from the memory.
BIO_new_mem_buf((char *){YOUR STRING},-1);

I was able to read the string from the certificate. But the problem was that
when i pass if to SSL_CTX_use_certificate_chain_file it fails.

Please send me any info if you have found a solution already

Thanks
Rakesh

SSLcontext with BSD Sockets

Xtremejames183's picture

Hi all,
Great article but i want to know how to attach SSL Context to BSD sockets(socket(),bind(),listen(),connect()..)
Thanks for help

Nice work, Eric. Thank

Anonymous's picture

Nice work, Eric. Thank you.

Just a comment: You looks like John Petrucci (guitarist of Dream Theater band)

Thanks for your nice

duanjigang's picture

Thanks for your nice artical, i am learning it

freeing of returned resource in check_cert()

manaz's picture

nice article, but there is a little memory leak in the check_cert() function. pointer returned from SSL_get_peer_certificate must be freed according to man page:

The reference count of the X509 object is incremented by one, so that it will not be destroyed when the session containing the peer certificate is freed. The X509 object must be explicitly freed using X509_free().

Re: An Introduction to OpenSSL Programming, Part I of II

Anonymous's picture

Nice article

need more info

Student's picture

can u please send me the link of where the next part is ?? that is part 2 of 2 ... thanks for the wonderful tutorial

Re: An Introduction to OpenSSL Programming, Part I of II

Anonymous's picture

huh!! nice..........

very good , makes it look all

Anonymous's picture

very good , makes it look all so easy! now to put it into practice

Webinar
One Click, Universal Protection: Implementing Centralized Security Policies on Linux Systems

As Linux continues to play an ever increasing role in corporate data centers and institutions, ensuring the integrity and protection of these systems must be a priority. With 60% of the world's websites and an increasing share of organization's mission-critical workloads running on Linux, failing to stop malware and other advanced threats on Linux can increasingly impact an organization's reputation and bottom line.

Learn More

Sponsored by Bit9

Webinar
Linux Backup and Recovery Webinar

Most companies incorporate backup procedures for critical data, which can be restored quickly if a loss occurs. However, fewer companies are prepared for catastrophic system failures, in which they lose all data, the entire operating system, applications, settings, patches and more, reducing their system(s) to “bare metal.” After all, before data can be restored to a system, there must be a system to restore it to.

In this one hour webinar, learn how to enhance your existing backup strategies for better disaster recovery preparedness using Storix System Backup Administrator (SBAdmin), a highly flexible bare-metal recovery solution for UNIX and Linux systems.

Learn More

Sponsored by Storix