Indy FTP, large files and NAT routers

Question

I have been using Indy to transfer files via FTP for many years, but could not find a satisfactory solution for the following problem.

When a user downloads a large file, behind the router, sometimes the following happens: the file is loaded in order, but at the same time the command channel is disconnected due to a timeout. Usually this does not happen with a direct connection to the server, because the server "knows" that the transfer is occurring on the data channel. Some routers are unaware of this, and the command channel is closed.

Many programs periodically send a NOOP command to support the command channel, even if this is not part of the standard FTP specification. My question is: how do I do this? Send NOOP command in OnWork event? Does this have any collateral damage in any way, for example, do I need to process some response? What is the best way to solve this problem?

Answer 1

We use several approaches for this: (1) Enable TCP / IP keepalives on the control channel during transmission and (2) restores gracefully after the connection drops, (3) support for the resumption of intermittent transfers.

Many FTP clients will send NOOP until everything works, but I don’t know if they send them during data transfer, because you will have to process the responses in this case, and many servers will not send them back until the data is completed .

• Indy 10.5.8 (Delphi XE2) supports TCP / IP natively. Just use the TIdFTP NATKeepAlive property.

  For previous releases, assign OnDataChannelCreate / OnDataChannelDestroy events:

   const KeepAliveIdle = 2 * SecsPerMin; KeepAliveInterval = 2 * SecsPerMin; IOC_VENDOR = $18000000; SIO_KEEPALIVE_VALS = DWORD(IOC_IN or IOC_VENDOR or 4); type tcp_keepalive = record onoff: u_long; keepalivetime: u_long; keepaliveinterval: u_long; end; procedure TFtpConnection.DataChannelCreated(Sender: TObject; ADataChannel: TIdTCPConnection); var Socket: TIdSocketHandle; ka: tcp_keepalive; Bytes: DWORD; begin // Enable/disable TCP/IP keepalives. They're very small (40-byte) packages // and will be sent every KeepAliveInterval seconds after the connection has // been idle for KeepAliveIdle seconds. In Win9x/NT4 the idle and timeout // values are system wide and have to be set in the registry; the default is // idle = 2 hours, interval = 1 second. Socket := (FIdFTP.IOHandler as TIdIOHandlerSocket).Binding; if Win32MajorVersion >= 5 then begin ka.onoff := 1; ka.keepalivetime := KeepAliveIdle * MSecsPerSec; ka.keepaliveinterval := KeepAliveInterval * MSecsPerSec; WSAIoctl(Socket.Handle, SIO_KEEPALIVE_VALS, @ka, SizeOf(ka), nil, 0, @Bytes, nil, nil) end else Socket.SetSockOpt(Id_SOL_SOCKET, Id_SO_KEEPALIVE, Id_SO_True) end; procedure TFtpConnection.DataChannelDestroy(ASender: TObject; ADataChannel: TIdTCPConnection); var Socket: TIdSocketHandle; begin Socket := (FIdFTP.IOHandler as TIdIOHandlerSocket).Binding; Socket.SetSockOpt(Id_SOL_SOCKET, Id_SO_KEEPALIVE, Id_SO_False) end; 

• To restore gracefully if the file was successfully migrated, simply reconnect it at the end and run SIZE or LIST to get the file size. If they match, the file was successfully migrated and you do not need to do anything. If the server supports it, you can also send an XCRC command to get the CRC value so that you can compare it with the local file.

• If you want to be truly reliable, you can also check out TIdFTP.CanResume . If it installs the server, it supports the REST command, so you can choose the transfer where you left off by passing true to the AResume parameter TIdFTP.Get/Put .