CarrierWave_Direct gem: keep existing directory structure on S3

Question

so I have a site already created using a camcorder with images stored on amazon s3. my bootloader uses the store_dir method to specify a specific structure for placing processed images.

well, now in my dev environment, I added the carrierwave_direct bug to start downloading directly to S3. The problem is that this pearl completely overrides the default store_dir and filename parameters in my bootloader. I cannot push my fully working bootloader alive because all of my old image links will be broken.

I understand that the CWdirect stone will upload the raw image file to the "temp" directory on S3, then S3 will respond and provide you with the key variable so that you can capture this file and process it as you like. therefore, should I use a completely separate carrier wave image downloader class to process the images and place them in the correct folders? that I will have one carrierwave_direct dedicated bootloader that downloads wherever this pearl seems to want to load; and I will use another uploader.rb class related to my real model, which saves the current store_dir structure and file name?

In any case, my main question is: how can I use the CarrierWave_Direct gem if I already have CW working with images in a specific folder structure?

Answer 1

Ok, so I figured out how to do this, and I will explain below. My guess was correct to use two different CarrierWave bootloader classes - one class designed to boot onto S3 (using the CarrierWave_Direct gem), and a second class used only for image processing (the class I already used in production). I will try to post the appropriate code below, but if anyone has any questions let me know. I'm not sure why I have not seen others using separate classes like this, but it seems to work for me.

My image Uploader class app\uploaders\image_uploader.rb using media: wavewave_direct gem:

 class ImageUploader < CarrierWave::Uploader::Base include CarrierWaveDirect::Uploader include ActiveModel::Conversion extend ActiveModel::Naming include CarrierWave::MimeTypes process :set_content_type # Add a white list of extensions which are allowed to be uploaded. # For images you might use something like this: def extension_white_list %w(jpg jpeg gif png) end # Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility: include Sprockets::Helpers::RailsHelper include Sprockets::Helpers::IsolatedHelper # Override the directory where uploaded files will be stored. # CarrierWaveDirect::Uploader puts raw uploaded files in this directory on S3 as a first step def store_dir "unprocessed_uploads" end end 

** Please note that processing is not performed in this class.

My image PROCESSING the class app\uploaders\image_processor.rb (which was already in production):

 class ImageProcessor < CarrierWave::Uploader::Base # Include RMagick or MiniMagick support: # include CarrierWave::RMagick include CarrierWave::MiniMagick include CarrierWave::MimeTypes process :set_content_type # Include the Sprockets helpers for Rails 3.1+ asset pipeline compatibility: include Sprockets::Helpers::RailsHelper include Sprockets::Helpers::IsolatedHelper # Choose what kind of storage to use for this uploader: storage :fog # Override the directory where uploaded files will be stored. # This is a sensible default for uploaders that are meant to be mounted: def store_dir "uploads/#{model.class.to_s.underscore}/#{mounted_as}/#{model.id}" # "uploads/#{model.class.to_s.underscore}/path/#{model.id}" end # Provide a default URL as a default if there hasn't been a file uploaded: def default_url "logos/" + [version_name, "default.png"].compact.join('_') end # Process files fetched from S3 after they are uploaded: def make_thumbnail(width, height) # uses MiniMagick classes to get a square, centered thumbnail image manipulate! do |img| if img[:width] < img[:height] remove = ((img[:height] - img[:width])/2).round img.shave("0x#{remove}") elsif img[:width] > img[:height] remove = ((img[:width] - img[:height])/2).round img.shave("#{remove}x0") end img.resize("#{width}x#{height}") img end end # Create different versions of your uploaded files: # the process statement below isn't defined within a version block on purpose--this means the ORIGINAL uploaded photo is constrained to 1050 pics process :resize_to_limit => [1050, 1050] process :quality => 85 # this reduces filesize greatly and saves space version :thumb do process :make_thumbnail => [100, 100] process :quality => 85 # this reduces filesize greatly and saves space end version :big_thumb do process :make_thumbnail => [350, 350] process :quality => 85 # this reduces filesize greatly and saves space end # Add a white list of extensions which are allowed to be uploaded. # For images you might use something like this: def extension_white_list %w(jpg jpeg gif png) end # Override the filename of the uploaded files: # Avoid using model.id or version_name here, see uploader/store.rb for details. def filename if original_filename if model && model.read_attribute(:image).present? model.read_attribute(:image) else "#{secure_token}.#{file.extension}" end end end protected def secure_token var = :"@#{mounted_as}_secure_token" model.instance_variable_get(var) or model.instance_variable_set(var, SecureRandom.uuid) end end 

My fashion model (in short):

 class Photo < ActiveRecord::Base mount_uploader :image, ImageProcessor def save_and_process_image(options = {}) s3_unprocessed_image_url = self.image.asset_host + '/' + self.key # this next line downloads the image from S3 # and this save line below will process the image and reupload to S3 according to ImageProcessor settings self.remote_image_url = s3_unprocessed_image_url save end end 

I also have a photo controller and a viewing code if I want it to let me know. I mainly use the ImageUploader class to bootstrap on S3 to a folder called unprocessed_uploads. Then S3 responds with the key field in the URL that I pass to the ImageProcessor class - it is attached to the Photo and processes the thumbnail and other images, and then reloads them into my uploads folder on S3.

This separation meant that I did not need to change the current folder structure to S3 when adding the carrierwave_direct gem. Hope this helps others. Let me know if you need more code. I'm tired of typing :)

UPDATE - adding more code:

Photo controller:

 class PhotosController < ApplicationController def index @photos = @photos.sort_by(&:created_at) @uploader = ImageUploader.new @uploader.success_action_redirect = new_tank_photo_url(@tank) respond_to do |format| format.html # index.html.erb format.json { render json: @photos } end end def create respond_to do |format| if @photo.save_and_process_image format.html { redirect_to tank_photos_path(@tank), notice: 'Photo uploaded successfully and is being processed...' } format.json { render json: @photo, status: :created, location: @photo } else format.html { render :new } format.json { render json: @photo.errors, status: :unprocessable_entity } end end end 

Photos Type of pointer, form with "Download" button:

  <%= direct_upload_form_for @uploader, :html => {:class => "form-inline"} do |f| %> <%= f.file_field :image %> <%= f.submit "Upload", :class => "btn btn-primary btn-medium" %> <% end %> 

Thus, with the above view / controller code, a brief description of the steps taken is provided. Note the difference between the ImageUploader class and the ImageProcessor class:

• In my photo controller, I create the @uploader variable, which has the ImageUploader class - this means that the images presented in my index / view form, which uses @uploader, upload the image to the "temp" folder. Because it uses then the ImageUploader class is not processed when this image is loaded (for now).
• When the user clicks the "Download" button on the form, the action Photo → create is called. This action calls @ photo.save_and_process_image - look at the model to see that this method actually captures the recently downloaded image from the S3 folder 'temp', processes it, and then reloads the processed images to the final destination. This is all possible, because my photomodule is associated with the ImageProcessor class, which performs processing / loading. It is not associated with a separate ImageUploader class.

Hope this helps explain what is going on with me.