An Introduction to OpenSSL Programming, Part II of II

Part two of the series that started in the September 2001 issue.
Write

When we're writing to the network we have to face the same sort of inconsistency that we had when reading. Again, the problem is the all-or-nothing nature of SSL record transmission. For simplicity's sake, let's consider the case where the network buffers are mostly full and the program attempts to perform a modest-sized write, say 1K. This is illustrated in Figure 2.

Figure 2. Write Interaction with SSL

Again, the left-hand side of the figure represents our initial situation. The program has 1K to write in some buffer. The write pointer is set at the beginning of that buffer. The SSL buffers are empty. The network buffer is half-full (the shading indicates the full region). The write pointer is set at the beginning. We've deliberately obscured the distinction between the TCP buffers and the size of the TCP window because it's not relevant here. Suffice to say that the program can safely write 512 bytes without blocking.

Now, the program calls SSL_write() with a 1024-byte block. OpenSSL has no way of knowing how much data it can write safely, so it simply formats the buffer as a single record, thus moving the write pointer in the program buffer to the end of the buffer. We can ignore the slight data expansion from the SSL header and MAC, and simply act as if the data to be written to the network was 1024 bytes.

Now, what happens when OpenSSL calls write()? It successfully writes 512 bytes but gets a would_block error when it attempts to write to the end of the record. As a consequence, the write pointer in the SSL buffer is moved halfway across--indicating that half of the data has been written to the network. The network buffer is shaded to indicate that it's completely full. The network write pointer hasn't moved.

We now need to concern ourselves with two questions: first, how does the toolkit indicate this situation to the application, and, second, how does the programmer arrange it so the SSL buffer gets flushed when space is available in the network buffer? The kernel will automatically flush the network buffer when possible, so we don't need to worry about arranging for that. We can use select() to see when there is more space available in the network buffer, and we should therefore flush the SSL buffer.

Once OpenSSL has received a would_block error from the network, it aborts and propagates that error all the way up to the application. Note that this does not mean it throws away the data in the SSL buffer. This is impossible because part of the record might already have been sent. In order to flush this buffer, we must call SSL_write() again with the same buffer that it called the first time (it's permissible to extend the buffer but the start must be the same.) OpenSSL automatically remembers where the buffer write pointer was and only writes the data after the write pointer. Listing 9 shows this process in action.

Listing 9. Client to Server Writes Using OpenSSL

98        /* Check for input on the console*/
99        if(FD_ISSET(fileno(stdin),&readfds)){
100          c2sl=read(fileno(stdin),c2s,BUFSIZZ);
101          if(c2sl==0){
102            shutdown_wait=1;
103            if(SSL_shutdown(ssl))
104              return;
105          }
106          c2s_offset=0;
107        }
108
109        /* If the socket is writeable... */
110        if((FD_ISSET(sock,&writefds) && c2sl) ||
111          (write_blocked_on_read && FD_ISSET(sock,&readfds))) {
112          write_blocked_on_read=0;
113
114          /* Try to write */
115          r=SSL_write(ssl,c2s+c2s_offset,c2sl);
116
117          switch(SSL_get_error(ssl,r)){
118            /* We wrote something*/
119            case SSL_ERROR_NONE:
120              c2sl-=r;
121              c2s_offset+=r;
122              break;
123
124              /* We would have blocked */
125            case SSL_ERROR_WANT_WRITE:
126              break;
127
128              /* We get a WANT_READ if we're
129                 trying to rehandshake and we block on
130                 write during the current connection.
131
132                 We need to wait on the socket to be readable
133                 but reinitiate our write when it is */
134            case SSL_ERROR_WANT_READ:
135              write_blocked_on_read=1;
136              break;
137
138                /* Some other error */
139            default:
140              berr_exit("SSL write problem");
141          }
142        }

The first thing we need to do is have some data to write. Thus, we check to see if the console is ready to read, and if so, read whatever's there (up to BUFSIZZ bytes) into the buffer c2s, placing the length in the variable c2sl.

If c2sl is nonzero and the network buffers are (at least partially) empty, then we have data to write to the network. As usual, we call SSL_write() with the buffer c2s. As before, if we manage to write some but not all of the data, we simply increment c2s_offset and decrement c2sl.

The new behavior here is that we check for the error SSL_ERROR_WANT_WRITE. This error indicates that we've got unflushed data in the SSL buffer. As we described above, we need to call SSL_write() again with the same buffer, so we simply leave c2sl and c2s_offset unchanged. Thus, the next time SSL_write() is called it will automatically be with the same data.

OpenSSL actually provides a flag called SSL_MODE_ACCEPT_MOVING_WRITE_BUFFER that allows you to call SSL_write() with a different buffer after a would_block error. However, this merely allows you to allocate a new buffer with the same contents. SSL_write() still seeks to the same write pointer before looking for new data.

______________________

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

Block socket with SSL_MODE_AUTO_RETRY

Anonymous's picture

If I use block socket with SSL_MODE_AUTO_RETRY
The flag SSL_MODE_AUTO_RETRY will cause read/write operations to only return after the handshake and successful completion.

Do I have to handle the retrying in SSL_Read and SSL_Write? Isn't it easier to do this way? I know it hurts throughput a little, but is it a big deal?

White Paper
Linux Management with Red Hat Satellite: Measuring Business Impact and ROI

Linux has become a key foundation for supporting today's rapidly growing IT environments. Linux is being used to deploy business applications and databases, trading on its reputation as a low-cost operating environment. For many IT organizations, Linux is a mainstay for deploying Web servers and has evolved from handling basic file, print, and utility workloads to running mission-critical applications and databases, physically, virtually, and in the cloud. As Linux grows in importance in terms of value to the business, managing Linux environments to high standards of service quality — availability, security, and performance — becomes an essential requirement for business success.

Learn More

Sponsored by Red Hat

White Paper
Private PaaS for the Agile Enterprise

If you already use virtualized infrastructure, you are well on your way to leveraging the power of the cloud. Virtualization offers the promise of limitless resources, but how do you manage that scalability when your DevOps team doesn’t scale? In today’s hypercompetitive markets, fast results can make a difference between leading the pack vs. obsolescence. Organizations need more benefits from cloud computing than just raw resources. They need agility, flexibility, convenience, ROI, and control.

Stackato private Platform-as-a-Service technology from ActiveState extends your private cloud infrastructure by creating a private PaaS to provide on-demand availability, flexibility, control, and ultimately, faster time-to-market for your enterprise.

Learn More

Sponsored by ActiveState