Geek Answers Handbook

How to connect to a FTPS server with data connection using the same TLS session?

Environment: I use Sun Java JDK 1.8.0_60 on 64-bit Windows 7 using Spring Integration 4.1.6 (which is supposed to use Apache Commons Net 3.3 to access FTPS).

I am trying to integrate automatic downloading from our FTPS client server with our application. I successfully managed SFTP servers using Spring Integration without any problems for other clients without problems, but this is the first time the client has demanded that we use FTPS and connecting to it was very puzzling. Although in my real application I configure Spring Integration using XML beans to try to understand what is not working, I use the following test code (although I will anonymize the actual host / username / password here):

final DefaultFtpsSessionFactory sessionFactory = new DefaultFtpsSessionFactory(); sessionFactory.setHost("XXXXXXXXX"); sessionFactory.setPort(990); sessionFactory.setUsername("XXXXXXX"); sessionFactory.setPassword("XXXXXXX"); sessionFactory.setClientMode(2); sessionFactory.setFileType(2); sessionFactory.setUseClientMode(true); sessionFactory.setImplicit(true); sessionFactory.setTrustManager(TrustManagerUtils.getAcceptAllTrustManager()); sessionFactory.setProt("P"); sessionFactory.setProtocol("TLSv1.2"); sessionFactory.setProtocols(new String[]{"TLSv1.2"}); sessionFactory.setSessionCreation(true); sessionFactory.setCipherSuites(new String[]{"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256"}); final FtpSession session = sessionFactory.getSession(); //try { final FTPFile[] ftpFiles = session.list("/"); logger.debug("FtpFiles: {}", (Object[]) ftpFiles); //} catch (Exception ignored ) {} session.close();

I run this code with -Djavax.net.debug=all to print all the TLS debugging information.

The main “control” connection to the FTPS server works fine, but when it tries to open a data connection (or any other data connection I tried), I get javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake , java.io.EOFException: SSL peer shut down incorrectly by java.io.EOFException: SSL peer shut down incorrectly . If I uncomment the catch-exceptions block around the session.list command, then I can see (although the output is javax.net.debug) that the server sent the following message after rejecting the SSL connection to connect to the data:

 main, READ: TLSv1.2 Application Data, length = 129 Padded plaintext after DECRYPTION: len = 105 0000: 34 35 30 20 54 4C 53 20 73 65 73 73 69 6F 6E 20 450 TLS session 0010: 6F 66 20 64 61 74 61 20 63 6F 6E 6E 65 63 74 69 of data connecti 0020: 6F 6E 20 68 61 73 20 6E 6F 74 20 72 65 73 75 6D on has not resum 0030: 65 64 20 6F 72 20 74 68 65 20 73 65 73 73 69 6F ed or the sessio 0040: 6E 20 64 6F 65 73 20 6E 6F 74 20 6D 61 74 63 68 n does not match 0050: 20 74 68 65 20 63 6F 6E 74 72 6F 6C 20 63 6F 6E the control con 0060: 6E 65 63 74 69 6F 6E 0D 0A nection..

Something is happening (and this is my first experience with FTPS, although I used to deal with simple FTP) that the way the server provides authentication and encryption via control and data transmission channels, this is what is normal after "A TLS connection to establish a control connection and authentication occurs there, each data connection requires that the client connect to the same TLS session. It makes sense to me how it should work, but the implementation of Apache Commons Net FTPS does not seem to do this. It seems to be trying to establish a new TLS session, and therefore the server rejects the attempt.

Based on this question about resuming SSL sessions in JSSE , it looks like Java assumes or requires a different session for each combination of hosts and messages. My hypothesis is that since the FTPS data connection is on a different port than the control connection, it does not find an existing session and tries to establish a new one, so the connection fails.

I see three main possibilities:

  • The server does not follow the FTPS standard, requiring the same TLS session on the data port as on the management port. I can connect to the server perfectly (using the same host / user / password that I am trying to use in my code) using FileZilla 3.13.1. The server identifies itself as “FileZilla Server 0.9.53 beta” at login, so maybe this is some kind of proprietary way of FileZilla to do something, and there is something strange I need to do to convince Java to use one the same TLS session.
  • The Apache Commons Net client does not actually comply with the FTPS standard and only allows some subsets that do not allow a data connection. This would seem strange, as this is the standard way to connect to FTPS from Java.
  • I completely missed something and misunderstood it.

I would appreciate any direction you can provide on how to connect to this FTPS server. Thanks.

+8
java ftp filezilla ftps apache-commons-net
source share
3 answers

Indeed, some FTP (S) servers require a reuse of a TLS / SSL session to connect to data. This is a security measure by which the server can verify that the data connection is being used by the same client as the control connection.

Some links for shared FTP servers:

What can help you with the implementation is that the Cyberduck FTP (S) client supports reusing a TLS / SSL session and uses the Apache Commons Net library:

  • https://trac.cyberduck.io/ticket/5087 - Reusing a session key when connecting to data

  • See its FTPClient.java code (extends Commons Net FTPSClient ), in particular its override the _prepareDataSocket_ method :

     @Override protected void _prepareDataSocket_(final Socket socket) throws IOException { if(preferences.getBoolean("ftp.tls.session.requirereuse")) { if(socket instanceof SSLSocket) { // Control socket is SSL final SSLSession session = ((SSLSocket) _socket_).getSession(); if(session.isValid()) { final SSLSessionContext context = session.getSessionContext(); context.setSessionCacheSize(preferences.getInteger("ftp.ssl.session.cache.size")); try { final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache"); sessionHostPortCache.setAccessible(true); final Object cache = sessionHostPortCache.get(context); final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class); method.setAccessible(true); method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session); method.invoke(cache, String.format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort())).toLowerCase(Locale.ROOT), session); } catch(NoSuchFieldException e) { // Not running in expected JRE log.warn("No field sessionHostPortCache in SSLSessionContext", e); } catch(Exception e) { // Not running in expected JRE log.warn(e.getMessage()); } } else { log.warn(String.format("SSL session %s for socket %s is not rejoinable", session, socket)); } } } }

  • It seems that the _prepareDataSocket_ method _prepareDataSocket_ been added to Commons Net FTPSClient specifically to allow TLS / SSL session reuse:
    https://issues.apache.org/jira/browse/NET-426

    Internal reuse support has not yet been completed:
    https://issues.apache.org/jira/browse/NET-408

  • You will obviously need to override the Spring DefaultFtpsSessionFactory.createClientInstance() integration in order to return your own FTPSClient implementation with session reuse support.

The above solution no longer works with JDK 8u161.

According to JDK 8u161 Update Release Notes (and response from @Laurent ):

Added TLS session hash code and extended extended extension support

...

In case of compatibility problems, the application can disable the negotiation of this extension by setting the System property jdk.tls.useExtendedMasterSecret to false in the JDK

Ie, you can call this to fix the problem:

 System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

Although this should be considered as a workaround. I do not know the right solution.

+13
source share

To make Martin Prikril’s proposal for me, I had to store the key not only under socket.getInetAddress().getHostName() , but also under socket.getInetAddress().getHostAddress() . (The solution stolen from here .)

+2
source share

You can use this SSLSessionReuseFTPSClient class:

 import java.io.IOException; import java.lang.reflect.Field; import java.lang.reflect.Method; import java.net.Socket; import java.util.Locale; import javax.net.ssl.SSLSession; import javax.net.ssl.SSLSessionContext; import javax.net.ssl.SSLSocket; import org.apache.commons.net.ftp.FTPSClient; public class SSLSessionReuseFTPSClient extends FTPSClient { // adapted from: // https://trac.cyberduck.io/browser/trunk/ftp/src/main/java/ch/cyberduck/core/ftp/FTPClient.java @Override protected void _prepareDataSocket_(final Socket socket) throws IOException { if (socket instanceof SSLSocket) { // Control socket is SSL final SSLSession session = ((SSLSocket) _socket_).getSession(); if (session.isValid()) { final SSLSessionContext context = session.getSessionContext(); try { final Field sessionHostPortCache = context.getClass().getDeclaredField("sessionHostPortCache"); sessionHostPortCache.setAccessible(true); final Object cache = sessionHostPortCache.get(context); final Method method = cache.getClass().getDeclaredMethod("put", Object.class, Object.class); method.setAccessible(true); method.invoke(cache, String .format("%s:%s", socket.getInetAddress().getHostName(), String.valueOf(socket.getPort())) .toLowerCase(Locale.ROOT), session); method.invoke(cache, String .format("%s:%s", socket.getInetAddress().getHostAddress(), String.valueOf(socket.getPort())) .toLowerCase(Locale.ROOT), session); } catch (NoSuchFieldException e) { throw new IOException(e); } catch (Exception e) { throw new IOException(e); } } else { throw new IOException("Invalid SSL Session"); } } } }

And with openJDK 1.8.0_161:

We must install:

 System.setProperty("jdk.tls.useExtendedMasterSecret", "false");

according to http://www.oracle.com/technetwork/java/javase/8u161-relnotes-4021379.html

Added TLS session hash code and extended extended extended key support

In the event of compatibility issues, the application can disable the negotiation of this extension by setting the System jdk.tls.useExtendedMasterSecret property to false in the JDK

+2
source share

More articles:


All Articles