Fast copy of progress file

Question

I am writing an SDL application for Linux that works from the console (without an X server). One of my functions is a file copying mechanism that copies certain files from the hard drive to a USB flash device and shows the progress of this copy in the user interface. To do this, I use a simple while loop and copy files to 8kB to get the copy progress. The problem is that it is slow. I can copy a 100 MB file in almost 10 minutes, which is unacceptable.

How can I implement a faster copy of the file? I was thinking of some kind of asynchronous API that would read the file from the hard drive to the buffer and store the data on USB in a separate stream, but I don’t know if I have to implement this myself, because it does not seem like an easy task. Maybe you know some C ++ API / libraries that can do this for me? Or maybe some other, better method?

Answer 1

Do not synchronously update your interface with copy success, which will significantly slow down the work. You must run a copy of the file in a separate thread from the main user interface thread so that the copy of the file can run as quickly as possible without interfering with the responsiveness of your application. The user interface can then be updated at a natural rate (for example, at the refresh rate of your monitor).

You should also use a larger buffer size than 8 KB. Experiment, but I think you will get faster results with larger buffer sizes (for example, in the range of 64-128 KB).

So, it might look something like this:

#define BUFSIZE (64*1024) volatile off_t progress, max_progress; void *thread_proc(void *arg) { // Error checking omitted for expository purposes char buffer[BUFSIZE]; int in = open("source_file", O_RDONLY); int out = open("destination_file", O_WRONLY | O_CREAT | O_TRUNC); // Get the input file size struct stat st; fstat(in, &st); progress = 0; max_progress = st.st_size; ssize_t bytes_read; while((bytes_read = read(in, buffer, BUFSIZE)) > 0) { write(out, buffer, BUFSIZE progress += bytes_read; } // copy is done, or an error occurred close(in); close(out); return 0; } void start_file_copy() { pthread_t t; pthread_create(&t, NULL, &thread_proc, 0); } // In your UI thread repaint handler, use the values of progress and // max_progress 

Note that if you are sending a file to a socket instead of another file, you should instead use the sendfile(2) system call, which copies the file directly to kernel space without circular disconnection in user space. Of course, if you do this, you will not be able to get any information about the work done so that it is not always ideal.

For Windows systems, you should use CopyFileEx , which is efficient and provides you with a callback routine.

Answer 2

Let the OS do all the work:

• Associate the file with memory: mmap , will significantly speed up the reading process.
• Save it to a file using msync .