Select and ssl in python

Question

I have a server application using select.select () and now I'm trying to add SSL to it, however I get the following error when listening to raw sockets:

ValueError: file descriptor cannot be a negative integer (-1) 

so I decided that instead I use ssl streams returned by ssl.wrap_socket. However, it does not return any errors, but it does not work - I am not quite sure what the problem is, I did a lot of research and ran into messages with similar problems, but I have not found a solution for this yet.

Really appreciate any help.

Answer 1

As Marius noted, select.select () works with SSL sockets, I still don't know what caused my silent error, but I jumped when I thought it was SSL + select (). Thus, this question is answered.

Answer 2

Using SSL sockets with select() not as simple as it might seem at first glance. Although they work well with him in the sense that he does not throw an error when you give it, if you just use them like regular sockets, you will sooner or later encounter some kind of weirdness.

Since select() needs a file descriptor, it will receive the original socket. But even if the raw socket becomes readable, this does not mean that you will receive data from the SSL socket. You will need to use non-blocking sockets (which is good when using select() ) and just ignore it if it throws SSL_ERROR_WANT_READ (equivalent to SSL EWOULDBLOCK ).

Another problem is that if you write 2048 bytes to connect at the other end, select() returned at your end. But if you are only reading 1024 bytes from an SSL socket, it is possible that the SSL socket internally reads more data, and the next select() will not return, although there will be more data to read, possibly to block the connection. This is because the raw socket used by select() does not have any data since it is already in the SSL socket buffers.

The first solution that comes to mind is to read more data until it reads throws SSL_ERROR_WANT_READ , thereby emptying the buffer. However, if the other end generates data faster than you can process it, this will lead to starvation of all your other connections until it finishes generating data.

You can see how much buffered data the SSL socket is sslsock.pending() calling sslsock.pending() . Thus, the best approach is to first do a read for a certain amount of data, check the amount of data pending, and issue a second read for just that amount of data, thereby emptying the buffer without causing any additional reads.

The man page for SSL_pending() (the C function backstage) also says the following:

SSL_pending () only considers bytes from the TLS / SSL record that is currently being processed (if any). If the read_ahead flag of the SSL object is set, additional protocol bytes containing more TLS / SSL entries can be read; they are ignored by SSL_pending ()

From what I understand, this means that if read_ahead set, you will need to repeat the second step until SSL_pending() returns 0. I'm sure python does not set read_ahead , but it is better to be safe than sorry. I included a loop in the sample code.

I am not familiar with this, but something like this should work:

 # Put the SSL socket to non-blocking mode sslsock.setblocking(0) while True: r, w, e = select.select([sslsock], [], []) if sslsock in r: try: data = sslsock.recv(1024) except ssl.SSLError as e: # Ignore the SSL equivalent of EWOULDBLOCK, but re-raise other errors if e.errno != ssl.SSL_ERROR_WANT_READ: raise continue # No data means end of file if not data: break # Drain the SSL socket internal buffer. # If you want to remove the loop, make sure you don't call recv() # with a 0 length, since that could cause a read to the raw socket. data_left = sslsock.pending() while data_left: data += sslsock.recv(data_left) data_left = sslsock.pending() # Process the data process(data)