Geek Answers Handbook

What is the correct way to upload to FTP with retries from commons-net?

I am having problems with a server that would close the connection to the FTP controller, despite the fact that I am sending requests with ftpClient.setControlKeepAliveTimeout support (CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS); Also sometimes it just disappears from SocketException, and not the usual FTPConnectionClosedException. In general, FTP is a very dodgy protocol that I use a lot, and each server connected to it requires a little configuration, but it is very difficult.

I know that a million things that I could have done wrong, to my question, is there any solution with attempts already implemented on FTP in case you lose the control connection, since this is not something that should be shocking (sometimes proxies / firewalls just accidentally lose your connection). Or there is an even more elegant way to solve this problem.

I have something like this.

public void store(File fileToUpload) throws IOException, InterruptedException {
    String filename = fileToUpload.getName();
    int retries = 0;
    while (true) {
        try {
            ftpClient.storeFile(filename, inputStreamFactory.getInputStream(fileToUpload));
        } catch (FTPConnectionClosedException | SocketException exception) {
            LOGGER.debug("Control connection lost uploading {}, continuing.", filename);
        }

        // This sleep is because there an anti-malware in the servers which makes the file not to appear
        // available immediately after an upload
        LOGGER.debug("Waiting {} milliseconds for anti-malware protection to process file", WAIT_AFTER_UPLOAD_MILLISECONDS);
        threadWrapper.sleep(WAIT_AFTER_UPLOAD_MILLISECONDS);
        if (!ftpClient.isConnected()) {
            connect();
        }

        LOGGER.debug("Checking if {} is already uploaded", filename);
        if (ftpFileChecker.isFileCompleted(listFiles(null), filename, fileToUpload.length())) {
            // Note this is likely to happen every time since their server will close the control
            // connection quite fast and FTPClient uses it at the end of storeFile
            LOGGER.debug("File {} was uploaded correctly", filename);
            break;
        } else {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded");
            } else {
                LOGGER.info("File {} was not uploaded, retrying", filename);
            }
        }
    }
}

public void connect() throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            ftpClient = ftpClientFactory.createFtpClient();
            ftpClient.connect(server, FTP_PORT);
            if (!ftpClient.login(username, password)) {
                LOGGER.error("Login to FTP failed");
                throw new ConfigurationException("Login to FTP failed");
            }
            ftpClient.enterLocalPassiveMode();
            ftpClient.setControlKeepAliveTimeout(CONTROL_CONNECTION_KEEP_ALIVE_INTERVAL_SECONDS);
            ftpClient.setFileType(FTPClient.BINARY_FILE_TYPE);
            ftpClient.changeWorkingDirectory(uploadDir);
            break;
        } catch (FTPConnectionClosedException | SocketException exception) {
            if (++retries > MAX_RETRIES) {
                throw new RemoteTimeoutException("Could not upload file, max retries exceeded", exception);
            } else {
                LOGGER.info("Could not login, retrying");
            }
        }
        LOGGER.debug("Sleeping {} milliseconds before trying to reconnect", WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
        threadWrapper.sleep(WAIT_BETWEEN_CONNECT_RETRIES_MILLISECONDS);
    }
}

public FTPFile[] listFiles(String directory) throws IOException, InterruptedException {
    int retries = 0;
    while (true) {
        try {
            return ftpClient.listFiles(directory);
        } catch (FTPConnectionClosedException exception) {
            LOGGER.debug("Control connection lost when listing files, continuing");
        } catch (SocketException exception) {
            LOGGER.debug("Socket exception when listing files, continuing");
        }
        if (!ftpClient.isConnected()) {
            connect();
        }
        if (++retries > MAX_RETRIES) {
            throw new RemoteTimeoutException("Could not list files, max retries exceeded");
        } else {
            LOGGER.info("Could not list files, retrying");
        }
    }
}

SocketException Broken Pipe listFiles, , , (, Windows NT -:( - , , -, , 5 , , , ).

VFS FTP-, , , , , (, ftp4j) maven, , .

.

: , , , , - .

+4
1

Spring, Spring . , Maven:

    <dependency>
        <groupId>org.springframework.retry</groupId>
        <artifactId>spring-retry</artifactId>
        <version>1.1.2.RELEASE</version>
    </dependency>
-1

More articles:


All Articles