Geek Answers Handbook

Proftpd check full download

I was wondering if there was a best practice for checking if the upload to your ftp server was successful.

The system I'm working with has a download directory that contains subdirectories for each user into which the files are downloaded.

Files in these directories are temporary, they are deleted after access.

The system goes through each of these subdirectories and the new files in them, and for each file it checks to see if it has been changed for 10 seconds. If it was not changed within 10 seconds, the system suggested that the file was downloaded successfully.

I do not like the way the system currently handles these situations because it will try to process the file and fail if the file download was incomplete, instead of waiting and letting the user resume the download before it finishes. This may be good for small files that do not require a lot of time to download, but if the file is large, I would like to resume downloading.

I also don't like directory and file loops, the system is idle with high CPU usage, so I implemented pyinotify to initiate an action when writing a file. I really did not look at the source code, I can only assume that it is more optimized than the current implementation (which does more than I described).

However, I still need to check if the file was downloaded successfully.

I know I can parse xferlog to get all full downloads. For instance:

awk '($12 ~ /^i$/ && $NF ~ /^c$/){print $9}' /var/log/proftpd/xferlog

This would make pyinotify unnecessary, since I can get the path for full and incomplete downloads if I only delay the log.

So, my solution would be to check xferlog in my run-loop and only process complete files.

If there is no best practice or just the best way to do this?

What would be the disadvantages of this method?

I run my application on a debian server, and proftpd is installed on the same server. In addition, I do not control clients sending the file.

+4
source share
2 answers

Looking at proftpd docs, I see http://www.proftpd.org/docs/directives/linked/config_ref_HiddenStores.html

The HiddenStores directive includes two-step file downloads: files are downloaded as ".in.filename". and as soon as the download is complete, it simply renamed "file name". This provides a degree of atomicity and helps prevent 1) incomplete downloads and 2) files still used in the download process.

This should be the "best way" to solve the problem when you control proftpd, since it handles all the work for you - you can assume that any file that does not run .in. , is a complete download. You can also safely delete any .in.* Orphan files after some arbitrary period of inactivity in script order.

+6
source

You can use pure-uploadscript if your pure-ftpd build has been compiled with --with-uploadscript .

It is used to run the specified script after each download is complete.

  • Set CallUploadScript to "yes"
  • Create a script using a command like touch /tmp/script.sh

  • Write the code. In my example, the script renames the file and adds ".completed" in front of the file name:

    #!/bin/bash fullpath=$1 filename=$(basename "$1") dirname=${fullpath%/*} mv "$fullpath" "$dirname/completed.$filename"

  • Run chmod 755 /tmp/script.sh to execute the pure-uploadscript executable script

  • Then run the command pure-uploadscript -B -r /etc/pure-ftpd/uploadscript.sh

Now /tmp/script.sh will be launched after each completed download.

+1
source

More articles:


All Articles