Upload file uploaded to resque process handler

Question

I just started using resque to do some processing on some very large files in the background, and it's hard for me to figure out how to pass the file to the resque worker. I use rails to handle file uploads, and rails create an object ActionDispatch::Http::UploadedFilefor each file loaded from the form.

How to send this file to a search worker? I tried sending the user hash only the path to the temporary file and the original file name, but I can no longer open the temporary file again in the resque work resource (just plain Errno::ENOENT - No such file or directory), as the rails seem to delete this temporary file after the request is complete.

Answer 1

Http::UploadedFileunavailable after request completion. You need to write the file somewhere (or use s3 as temporary storage). Skip resque path to the file you wrote.

Answer 2

I just spent two days trying to do this, and finally figured it out. You need Base64 to encode the file so that it can be serialized in json. Then you need to decode it in the working file and create a new

ActionDispatch::Http::UploadedFile

Here's how to encode and pass in resque:

// You only need to encode the actual file, everything else in the 
// ActionDispatch::Http::UploadedFile object is just string or a hash of strings

file = params[:file] // Your ActionDispatch::Http::UploadedFile object 
file.tempfile.binmode
file.tempfile = Base64.encode64(file.tempfile.read)

Resque.enqueue(QueueWorker, params)  

And here's how to decode and convert back to an object inside your worker

class QueueWorker
    @queue = :main_queue

    def self.perform(params)
        file = params['file']
        tempfile = Tempfile.new('file')
        tempfile.binmode
        tempfile.write(Base64.decode64(file['tempfile']))

        // Now that the file is decoded you need to build a new
        // ActionDispatch::Http::UploadedFile with the decoded tempfile and the other
        // attritubes you passed in.

        file = ActionDispatch::Http::UploadedFile.new(tempfile: tempfile, filename: file['original_filename'], type: file['content_type'], head: file['headers'])

        // This object is now the same as the one in your controller in params[:file]
    end
end